Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Platform API ref follow-on updates, part 1 #404

Merged
merged 1 commit into from
Dec 18, 2024
Merged

Platform API ref follow-on updates, part 1 #404

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 22 additions & 23 deletions platform/api/overview.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,12 41,12 @@ To use the Unstructured Platform API, you must have:
Unstructured Platform API. See the next bullet for the Unstructured Platform API URL.
</Warning>

- **The Unstructured Platform API URL**, which is `http://platform.unstructuredapp.io/api/v1`
- **The Unstructured Platform API URL**, which is `https://platform.unstructuredapp.io/api/v1`

<Note>
If you signed up through the [For Enterprise](https://unstructured.io/enterprise) page, your API URL
might be different. For API URL guidance, email Unstructured Sales at [[email protected]](mailto:[email protected]).
If your API URL is different, be sure to substitute `http://platform.unstructuredapp.io/api/v1` for your
If your API URL is different, be sure to substitute `https://platform.unstructuredapp.io/api/v1` for your
API URL throughout the following examples.
</Note>

Expand All @@ -55,6 55,11 @@ utilities, tools, programming languages, packages, and libraries. The following
`curl` and Postman. You can adapt this information as needed for your preferred programming languages and libraries, for example by using the
`requests` library with Python.

<Tip>
You can also use the [Unstructured Platform API - Swagger UI](https://platform.unstructuredapp.io/docs) to call the REST endpoints
that are available through `https://platform.unstructuredapp.io`.
</Tip>

<Note>
The Unstructured Platform API is separate from [Unstructured Serverless API services](/api-reference/api-services/overview) and
[Unstructured Ingest](/api-reference/ingest/overview).
Expand Down Expand Up @@ -94,10 99,15 @@ For general information about these objects, see:

The following sections provide examples, showing the use of `curl` or Postman, for all of the supported REST endpoints.

<Tip>
You can also use the [Unstructured Platform API - Swagger UI](https://platform.unstructuredapp.io/docs) to call the REST endpoints
that are available through `https://platform.unstructuredapp.io`.
</Tip>

The following `curl` examples use environment variables, which you can set as follows:

```bash
export UNSTRUCTURED_API_URL="http://platform.unstructuredapp.io/api/v1"
export UNSTRUCTURED_API_URL="https://platform.unstructuredapp.io/api/v1"
export UNSTRUCTURED_API_KEY="<your-unstructured-api-key>"
```

Expand All @@ -112,8 122,8 @@ The following Postman examples use variables, which you can set as follows:

- **Variable**: `UNSTRUCTURED_API_URL`
- **Type**: `default`
- **Initial value**: `http://platform.unstructuredapp.io/api/v1`
- **Current value**: `http://platform.unstructuredapp.io/api/v1`
- **Initial value**: `https://platform.unstructuredapp.io/api/v1`
- **Current value**: `https://platform.unstructuredapp.io/api/v1`
<br/>
- **Variable**: `UNSTRUCTURED_API_URL`
- **Type**: `secret`
Expand Down Expand Up @@ -545,14 555,13 @@ To list workflows, use the `GET` method to call the `/workflows` endpoint.

To filter the list of workflows, use one or more of the following query parameters:

- `source_connector_id=<connector-id>`, replacing `<connector-id>` with the source connector's unique ID.
- `source_id=<connector-id>`, replacing `<connector-id>` with the source connector's unique ID.
To get this ID, see [List source connectors](#list-source-connectors).
- `destination_connector_id=<connector-id>`, replacing `<connector-id>` with the destination connector's unique ID.
- `destination_id=<connector-id>`, replacing `<connector-id>` with the destination connector's unique ID.
To get this ID, see [List destination connectors](#list-destination-connectors).
- `workflow_type=<type>`, replacing `<type>` with one of the following workflow types: `advanced`, `basic`, or `platinum`.
- `status=<status>`, replacing `<status>` with one of the following workflow statuses: `active` or `inactive`.

You can specify multiple query parameters, for example `?source_connector_id=<connector-id>&status=<status>`.
You can specify multiple query parameters, for example `?source_id=<connector-id>&status=<status>`.

<AccordionGroup>
<Accordion title="curl">
Expand All @@ -567,7 576,7 @@ You can specify multiple query parameters, for example `?source_connector_id=<co

```bash
curl --request 'GET' --location \
"$UNSTRUCTURED_API_URL/workflows?source_connector_id=<connector-id>" \
"$UNSTRUCTURED_API_URL/workflows?source_id=<connector-id>" \
--header 'accept: application/json' \
--header "unstructured-api-key: $UNSTRUCTURED_API_KEY"
```
Expand All @@ -576,16 585,7 @@ You can specify multiple query parameters, for example `?source_connector_id=<co

```bash
curl --request 'GET' --location \
"$UNSTRUCTURED_API_URL/workflows?destination_connector_id=<connector-id>" \
--header 'accept: application/json' \
--header "unstructured-api-key: $UNSTRUCTURED_API_KEY"
```

To filter the list by workflow type:

```bash
curl --request 'GET' --location \
"$UNSTRUCTURED_API_URL/workflows?workflow_type=<type>" \
"$UNSTRUCTURED_API_URL/workflows?destination_id=<connector-id>" \
--header 'accept: application/json' \
--header "unstructured-api-key: $UNSTRUCTURED_API_KEY"
```
Expand Down Expand Up @@ -614,9 614,8 @@ You can specify multiple query parameters, for example `?source_connector_id=<co

4. To filter the list of workflows, on the **Params** tab, enter one or more of the following query parameter:

- By source connector ID: **Key**: `source_connector_id`, **Value**: `<connector-id>`
- By destination connector ID: **Key**: `destination_connector_id`, **Value**: `<connector-id>`
- By workflow type: **Key**: `workflow_type`, **Value**: `<type>`
- By source connector ID: **Key**: `source_id`, **Value**: `<connector-id>`
- By destination connector ID: **Key**: `destination_id`, **Value**: `<connector-id>`
- By workflow status: **Key**: `status`, **Value**: `<status>`

5. Click **Send**.
Expand Down
43 changes: 14 additions & 29 deletions platform/api/workflows.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 29,10 @@ specify the settings for the workflow, as follows:
--data \
'{
"name": "<name>",
"source_connector_id": "<source-connector-id>",
"destination_connector_id": "<destination-connector-id>",
"type": "<type>",
"schedule": "<schedule>"
"source_id": "<source-connector-id>",
"destination_id": "<destination-connector-id>",
"workflow_type": "<type>",
"schedule": "<cron-expression>"
}'
```
</Accordion>
Expand All @@ -54,10 54,10 @@ specify the settings for the workflow, as follows:
```json
{
"name": "<name>",
"source_connector_id": "<source-connector-id>",
"destination_connector_id": "<destination-connector-id>",
"type": "<type>",
"schedule": "<schedule>"
"source_id": "<source-connector-id>",
"destination_id": "<destination-connector-id>",
"workflow_type": "<type>",
"schedule": "<cron-expression>"
}
```

Expand All @@ -73,27 73,12 @@ Replace the preceding placeholders as follows:
- `<destination-connector-id>` (_required_) - The ID of the target destination connector. To get the ID,
use the `GET` method to call the `/destinations` endpoint. [Learn more](/platform/api/overview#list-destination-connectors).
- `<type>` (_required_) - The workflow optimization type. Available values include `advanced`, `basic`, and `platinum`.
- `<schedule>` - The repeating automatic run schedule. Available values include the following:

- `don't repeat` (_default_) - This workflow will not run on a repeating automatic schedule. To run the workflow manually,
Use the `POST` method to call the `/workflows/<workflow-id>/run` endpoint. [Learn more](/platform/api/overview#run-a-workflow).
- `every 15 minutes` - Run this workflow once every 15 minutes.
- `every hour` - Run this workflow once at the beginning of every hour.
- `every 2 hours` - Run this workflow once at the beginning of the hour every 2 hours.
- `every 4 hours` - Run this workflow once at the beginning of the hour every 4 hours.
- `every 6 hours` - Run this workflow once at the beginning of the hour every 6 hours.
- `every 8 hours` - Run this workflow once at the beginning of the hour every 8 hours.
- `every 10 hours` - Run this workflow once at the beginning of the hour every 10 hours.
- `every 12 hours` - Run this workflow once at the beginning of the hour every 12 hours.
- `daily` - Run this workflow once each day. To specify the time of day, ...
- `weekly` - Run this workflow once on each of the specified days of each week.
To specify the days of the week, ....
To specify the time for all of these days, ....
- `monthly` Run this workflow once on each of the specified days of each month.
To specify the days of the month, ....
To specify the time for all of these days, ....

The default value is `don't repeat` if not otherwise specified.
- `<cron-expression>` - The repeating automatic run schedule, specified as a cron expression, for example
`59 11 * * 4` to run the workflow at 11:59 PM every Thursday.
[Get help building cron expressions](https://crontab.guru/).
[Learn more](https://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/crontrigger.html).

If `schedule` is not specified or if `crontab_entries` is empty, the workflow does not automatically run on a repeating schedule.

## Update a workflow

Expand Down
2 changes: 0 additions & 2 deletions snippets/destination_connectors/astradb_rest_change.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 6,6 @@ curl --request 'PUT' --location \
--header 'content-type: application/json' \
--data \
'{
"name": "<name>",
"type": "astradb",
"config": {
"token": "<token>",
"api_endpoint": "<api-endpoint>",
Expand Down
2 changes: 0 additions & 2 deletions snippets/destination_connectors/azure_ai_search_rest_change.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 6,6 @@ curl --request 'PUT' --location \
--header 'content-type: application/json' \
--data \
'{
"name": "<name>",
"type": "azure_ai_search",
"config": {
"endpoint": "<endpoint>",
"index": "<index>",
Expand Down
2 changes: 0 additions & 2 deletions snippets/destination_connectors/databricks_volumes_rest_change.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 6,6 @@ curl --request 'PUT' --location \
--header 'content-type: application/json' \
--data \
'{
"name": "<name>",
"type": "databricks_volumes",
"config": {
"host": "<host>",
"catalog": "<catalog>",
Expand Down
12 changes: 6 additions & 6 deletions snippets/destination_connectors/delta_table_rest_change.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 6,11 @@ curl --request 'PUT' --location \
--header 'content-type: application/json' \
--data \
'{
"type": "delta_table",
"name": "<name>",
"aws_region": "<aws-region>",
"table_uri": "<table-uri>",
"aws_access_key_id": "<aws-access-key-id>",
"aws_secret_access_key": "<aws-secret-access-key>"
"config": {
"aws_region": "<aws-region>",
"table_uri": "<table-uri>",
"aws_access_key_id": "<aws-access-key-id>",
"aws_secret_access_key": "<aws-secret-access-key>"
}
}'
```
10 changes: 6 additions & 4 deletions snippets/destination_connectors/delta_table_rest_create.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 8,11 @@ curl --request 'POST' --location \
'{
"type": "delta_table",
"name": "<name>",
"aws_region": "<aws-region>",
"table_uri": "<table-uri>",
"aws_access_key_id": "<aws-access-key-id>",
"aws_secret_access_key": "<aws-secret-access-key>"
"config": {
"aws_region": "<aws-region>",
"table_uri": "<table-uri>",
"aws_access_key_id": "<aws-access-key-id>",
"aws_secret_access_key": "<aws-secret-access-key>"
}
}'
```
2 changes: 0 additions & 2 deletions snippets/destination_connectors/elasticsearch_rest_change.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 6,6 @@ curl --request 'PUT' --location \
--header 'content-type: application/json' \
--data \
'{
"name": "<name>",
"type": "elasticsearch",
"config": {
"hosts": ["<host-url>"],
"es_api_key": "<es-api-key>",
Expand Down
8 changes: 4 additions & 4 deletions snippets/destination_connectors/gcs_rest_change.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 6,9 @@ curl --request 'PUT' --location \
--header 'content-type: application/json' \
--data \
'{
"type": "gcs",
"name": "<name>",
"remote_url": "<remote-url>",
"service_account_key": "<service-account-key>"
"config": {
"remote_url": "<remote-url>",
"service_account_key": "<service-account-key>"
}
}'
```
6 changes: 4 additions & 2 deletions snippets/destination_connectors/gcs_rest_create.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 8,9 @@ curl --request 'POST' --location \
'{
"type": "gcs",
"name": "<name>",
"remote_url": "<remote-url>",
"service_account_key": "<service-account-key>"
"config": {
"remote_url": "<remote-url>",
"service_account_key": "<service-account-key>"
}
}'
```
4 changes: 1 addition & 3 deletions snippets/destination_connectors/milvus_rest_change.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 6,7 @@ curl --request 'PUT' --location \
--header 'content-type: application/json' \
--data \
'{
"name": "<name>",
"type": "milvus",
"config": {
"config": {
"user": "<user>",
"uri": "<uri>",
"db_name": "<db-name>",
Expand Down
2 changes: 0 additions & 2 deletions snippets/destination_connectors/mongodb_rest_change.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 6,6 @@ curl --request 'PUT' --location \
--header 'content-type: application/json' \
--data \
'{
"name": "<name>",
"type": "mongodb",
"config": {
"database": "<database>",
"collection": "<collection>",
Expand Down
2 changes: 0 additions & 2 deletions snippets/destination_connectors/onedrive_rest_change.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 6,6 @@ curl --request 'PUT' --location \
--header 'content-type: application/json' \
--data \
'{
"name": "<name>",
"type": "onedrive",
"config": {
"client_id": "<client-id>",
"user_pname": "<user-pname>",
Expand Down
2 changes: 0 additions & 2 deletions snippets/destination_connectors/pinecone_rest_change.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 6,6 @@ curl --request 'PUT' --location \
--header 'content-type: application/json' \
--data \
'{
"name": "<name>",
"type": "pinecone",
"config": {
"index_name" "<index-name>",
"api_key": "<api-key>",
Expand Down
2 changes: 0 additions & 2 deletions snippets/destination_connectors/s3_rest_change.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 6,6 @@ curl --request 'PUT' --location \
--header 'content-type: application/json' \
--data \
'{
"name": "<name>",
"type": "s3",
"config": {
# For AWS access key ID with AWS secret access key authentication:
"key": "<key>",
Expand Down
2 changes: 0 additions & 2 deletions snippets/destination_connectors/weaviate_rest_change.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 6,6 @@ curl --request 'PUT' --location \
--header 'content-type: application/json' \
--data \
'{
"name": "<name>",
"type": "weaviate",
"config": {
"host_url": "<host-url>",
"class_name": "<class-name>",
Expand Down
2 changes: 0 additions & 2 deletions snippets/source_connectors/azure_rest_change.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 6,6 @@ curl --request 'PUT' --location \
--header 'content-type: application/json' \
--data \
'{
"name": "<name>",
"type": "azure",
"config": {
"remote_url": "az://<container-name>/<path/to/file/or/folder>",
"recursive": <true|false>,
Expand Down
2 changes: 0 additions & 2 deletions snippets/source_connectors/confluence_rest_change.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 6,6 @@ curl --request 'PUT' --location \
--header 'content-type: application/json' \
--data \
'{
"name": "<name>",
"type": "confluence",
"config": {
"url": "<url>",
"user_email": "<user-email>",
Expand Down
2 changes: 0 additions & 2 deletions snippets/source_connectors/databricks_volumes_rest_change.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 6,6 @@ curl --request 'PUT' --location \
--header 'content-type: application/json' \
--data \
'{
"name": "<name>",
"type": "databricks_volumes",
"config": {
"host": "<host>",
"client_id": "<client-id>"
Expand Down
2 changes: 0 additions & 2 deletions snippets/source_connectors/gcs_rest_change.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 6,6 @@ curl --request 'PUT' --location \
--header 'content-type: application/json' \
--data \
'{
"name": "<name>",
"type": "gcs",
"config": {
"service_account_key": "<service-account-key>",
"remote_url": "<remote-url>",
Expand Down
2 changes: 0 additions & 2 deletions snippets/source_connectors/mongodb_rest_change.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 6,6 @@ curl --request 'PUT' --location \
--header 'content-type: application/json' \
--data \
'{
"name": "<name>",
"type": "mongodb",
"config": {
"uri": "<uri>",
"database": "<database>",
Expand Down
2 changes: 0 additions & 2 deletions snippets/source_connectors/onedrive_rest_change.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 6,6 @@ curl --request 'PUT' --location \
--header 'content-type: application/json' \
--data \
'{
"name": "<name>",
"type": "onedrive",
"config": {
"client_id": "<client-id>",
"user_pname": "<user-pname>",
Expand Down
2 changes: 0 additions & 2 deletions snippets/source_connectors/outlook_rest_change.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 6,6 @@ curl --request 'PUT' --location \
--header 'content-type: application/json' \
--data \
'{
"type": "outlook",
"name": "<name>",
"config": {
"client_id": "<client-id>",
"authority_url": "<authority-url>",
Expand Down
Loading