Microsoft Azure HDInsightJobManagementClient
Browse all Azure APIsAgent-ready OpenAPI document · curated by JenticDeveloper ToolsCi Cdoauth210 EndpointsREST
Know of an official OpenAPI document? Contribute it →
For Agents
Submit Hive, Pig, MapReduce, and Sqoop jobs to a running HDInsight cluster, list active jobs, fetch job state, and kill jobs.
Use for: I need to submit a Hive query to my HDInsight cluster, Submit a MapReduce streaming job using a Python mapper and reducer, List all jobs currently running on cluster 'analytics-prod', Get the status of job 'job_1696...' on the cluster
Not supported: Does not provision clusters, manage cluster nodes, or run Spark or Databricks workloads - use for submitting and managing Templeton/WebHCat jobs on a running HDInsight cluster only.
Jentic publishes the only available OpenAPI specification for HDInsightJobManagementClient, keeping it validated and agent-ready. The HDInsight Job Management API submits and tracks Hadoop jobs against a running HDInsight cluster via the cluster's WebHCat (Templeton) endpoint. It supports Hive, Pig, Sqoop, MapReduce, and MapReduce streaming job submissions, plus job listing and kill operations. The API runs against the cluster DNS name (e.g. clustername.azurehdinsight.net) and is authenticated with cluster gateway credentials over OAuth2.
Install Jentic One Beta
Connect the HDInsightJobManagementClient to your agent
Jentic One is a self-hosted execution layer for AI agents. It lets your agent call the HDInsightJobManagementClient, or any other public or private API you need. You set the rules, the agent never sees your credentials, and every call is logged.
Two steps, two machines. Install the instance in a safe environment, then register your agent from wherever it runs.
1
Step 1: Jentic One Host machine
# On the machine that will host your Jentic One instance:
curl -fsSL "https://jentic.com/install.sh?src=apis&api=%2Fapis%2Fazure.com%2Fhdinsightjobmanagementclient" | sh2
Step 2: Agent machine
# On the machine where your agent runs (keep this separate from the instance):
curl -fsSL "https://jentic.com/install.sh?src=apis&api=%2Fapis%2Fazure.com%2Fhdinsightjobmanagementclient" | sh
jentic register # connects your agent to your Jentic One instanceJentic One is in public beta. The setup above keeps your agent separate from the instance, which is what you want before using real credentials: an agent running as the same OS user as Jentic One can read its stored keys directly. Just evaluating? A single local install is fine to start. See the secure deployment guide for the tiers.
Capabilities
What an agent can do with HDInsightJobManagementClient API.
Submit a Hive job to an HDInsight cluster
Submit a MapReduce or MapReduce streaming job and track its job ID
Submit Pig and Sqoop jobs to the same cluster surface
List jobs running on the cluster, including those started after a given job ID
Retrieve the status and metadata of a specific job by job ID
Kill a running job to free cluster capacity
Inspect the YARN application state for a submitted job
Use Cases
Patterns agents use HDInsightJobManagementClient API for, with concrete tasks.
★ Submit Hive jobs from a data pipeline
Submit Hive queries to an HDInsight cluster from an orchestration step via POST /templeton/v1/hive. The API returns the WebHCat job ID, which can then be polled via GET /templeton/v1/jobs/{jobId} until the job reaches a terminal state. Useful inside scheduled ETL pipelines that already produce input data on the cluster's storage.
POST /templeton/v1/hive with a Hive query that aggregates orders.csv and poll /templeton/v1/jobs/{jobId} until the status is SUCCEEDED.
Run streaming MapReduce jobs
Submit a MapReduce streaming job using Python or shell mappers via POST /templeton/v1/mapreduce/streaming, supplying input and output paths on cluster storage. WebHCat hands the job to YARN and returns a job ID for tracking. Useful when teams have legacy streaming jobs that pre-date Spark.
POST /templeton/v1/mapreduce/streaming with mapper.py and reducer.py and the agreed input and output paths on the cluster's primary storage container.
Active job monitoring and kill switch
List running jobs via GET /templeton/v1/jobs, page through with the LISTAFTERID variant, and kill any job that exceeds its expected duration via DELETE /templeton/v1/jobs/{jobId}. Useful for SREs operating shared HDInsight clusters where runaway jobs can starve other workloads.
List jobs on the cluster, find any in RUNNING state for more than 2 hours, and DELETE /templeton/v1/jobs/{jobId} for each.
AI agent job orchestration via Jentic
A data engineering agent uses Jentic to find the right Job_Submit operation for a query, load its schema, and execute it without holding raw cluster credentials. Job IDs returned by submission are then re-fed into Job_Get for polling. Jentic injects the bearer token at execution time.
Use Jentic to search for 'submit a hive job to hdinsight', load Job_SubmitHiveJob, and execute it for the agreed query against cluster 'analytics-prod'.
Key Endpoints
10 endpoints — jentic publishes the only available openapi specification for hdinsightjobmanagementclient, keeping it validated and agent-ready.
METHOD
PATH
DESCRIPTION
POST
/templeton/v1/hive
Submit a Hive job
POST
/templeton/v1/mapreduce/jar
Submit a MapReduce job from a JAR
POST
/templeton/v1/mapreduce/streaming
Submit a MapReduce streaming job
POST
/templeton/v1/pig
Submit a Pig job
POST
/templeton/v1/sqoop
Submit a Sqoop job
GET
/templeton/v1/jobs/{jobId}
Get a specific job's status
DELETE
/templeton/v1/jobs/{jobId}
Kill a running job
GET
/ws/v1/cluster/apps/{appId}/state
Get YARN application state
POST
/templeton/v1/hive
Submit a Hive job
POST
/templeton/v1/mapreduce/jar
Submit a MapReduce job from a JAR
POST
/templeton/v1/mapreduce/streaming
Submit a MapReduce streaming job
POST
/templeton/v1/pig
Submit a Pig job
POST
/templeton/v1/sqoop
Submit a Sqoop job
GET
/templeton/v1/jobs/{jobId}
Get a specific job's status
DELETE
/templeton/v1/jobs/{jobId}
Kill a running job
GET
/ws/v1/cluster/apps/{appId}/state
Get YARN application state
Why Jentic?
What agents get from Jentic-routed access to this vendor.
Setup
Wiring HDInsight job submission by hand means learning the WebHCat (Templeton) URL conventions, targeting the cluster's own DNS host rather than management.azure.com, authenticating with cluster gateway credentials, and polling YARN job state yourself. Through Jentic you install once, import HDInsight Job Management from the API Directory, store the cluster gateway credentials once, and your agent calls it.
Permission scoping
Job submission endpoints such as /templeton/v1/hive and /templeton/v1/mapreduce/streaming carry no resource id in the path, while job lookup and kill use /templeton/v1/jobs/{jobId}. Limit the agent to the operations it needs, such as submitting a Hive job and getting job status, so killing a running job is not available unless you add it.
Credential isolation
Your HDInsight cluster gateway credentials are stored once, encrypted, by your own Jentic One instance and exchanged for a bearer token at execution time. The raw cluster password never enters the agent's prompt, logs, or context.
Intent-based discovery
Agents search Jentic by intent such as 'submit a hive job to an hdinsight cluster' or 'get hdinsight job status', and Jentic returns the matching Job operation with its input schema so the agent submits work without learning the Templeton URL layout.
Related APIs
Alternatives and complements available in the Jentic catalogue.
FAQs
Specific to using HDInsightJobManagementClient API through Jentic.
Why is there no official OpenAPI spec for HDInsightJobManagementClient?
Microsoft Azure does not publish a stand-alone OpenAPI specification for the HDInsight WebHCat job submission surface. Jentic generates and maintains this spec from the Templeton REST surface so that AI agents and developers can call HDInsightJobManagementClient via structured tooling. It is validated against the live API and kept up to date. Get started with Jentic One, the self-hosted execution layer.
What authentication does the HDInsight Job API use?
The API uses cluster gateway authentication over OAuth2, declared as the azure_auth security scheme. The host is the cluster DNS name (clustername.azurehdinsight.net), not management.azure.com. Through Jentic, the cluster gateway credentials are stored encrypted and exchanged for a bearer token at execution time.
Can I submit a Sqoop job through this API?
Yes. POST /templeton/v1/sqoop accepts a Sqoop command string and submits the job to the cluster. The response includes the WebHCat job ID, which can be polled at GET /templeton/v1/jobs/{jobId}.
What are the rate limits?
Limits are enforced by the cluster gateway and YARN scheduler rather than Azure ARM. Concurrent job count depends on cluster size and queue capacity - the API itself does not throttle, but the cluster will reject submissions when YARN is saturated.
How do I submit a Hive job through Jentic?
Run jentic.search('submit a hive job to an hdinsight cluster'), load the Job_SubmitHiveJob schema, and execute it with the cluster DNS name in the host parameter and your Hive query in the body. The response contains the WebHCat job ID for polling Job_Get.
Can I limit what my agent is allowed to do with the Azure HDInsight Job Management API?
Yes. Because you run Jentic One yourself, your own rules decide which HDInsight Job operations and credentials the agent may use. You can allow only the operations it needs, such as submitting a Hive job via POST /templeton/v1/hive and checking status via GET /templeton/v1/jobs/{jobId}, while withholding others. That means an operation like killing a running job through DELETE /templeton/v1/jobs/{jobId} stays unavailable unless you explicitly grant it.
GET STARTED