REST

While Skuid’s pre-configured data source types allow you to easily connect to a variety of systems, the REST data source type (DST) allows advanced builders to connect to services that provide a REST API. REST is web service architecture that typically uses the HTTP protocol, identifies resources with a URL, and represents actions with HTTP methods (also called verbs) such as GET, POST, PUT, and DELETE.

Skuid extends the capability to a wide range of REST services, but REST standards such as Swagger can be very helpful in creating services that are easy to use and understand.

Expectations for Skuid’s REST data source type

Skuid’s REST data source type is primarily intended for connecting to stateless services where either of the following can be accomplished with HTTP verbs:

  • Facilitating create-read-update-delete (CRUD) behaviors on tabular data
  • Activating webhooks for actions to be run on a server

Skuid assumes it has arbitrary, random access to the service and that responses are returned in consistent, parseable formats (JSON and XML). While connecting to stateful REST services may be possible, they may take additional builder/developer resources—and may not function as expected.

Concepts and Prerequisites

While the REST data source type is one of the most versatile, it is also one of the most complex. To utilize it fully, you need to have at least a basic-to-moderate understanding of the following concepts:

You’ll also need to be familiar with the following for your external system of choice:

  • The individual endpoints for each type of operation and their expected HTTP method
  • The expected request format (headers vs URL parameters vs request payloads)
  • The expected response format
  • Your chosen authentication method (OAuth, basic authentication, etc)

For more technical information, see RFC 9110 for HTTP semantics.

Configuration

To begin creating a new data source, click Configure > Data Sources > New Data Source.

Inside the New Data Source popup,

  1. Select an appropriate option under Data source type
  2. Give your data source a unique name.
  3. Click Next Step.

Configure data source connection and authentication settings

  • URL / Endpoint: This is the base URL of the data source. For REST services, this will be the first part of the URL—which should be the same for all endpoints in models pointed to this data source.

    As with the example above, this value will look similar to http://www.example.com/api/v1/ for most REST data sources.

  • Use Apex Proxy : Choose whether or not to use the Apex Proxy. By default, this option is checked. See here for more information on the implications of using the proxy.

  • Authentication Method, Provider, and Credential Source: Configure the appropriate authentication settings for this particular data source. Click here for more information about authentication in Skuid.

  • URL Parameters to send with every request: The request URL will append these parameters to each data source URL for each request. This is typically used for API keys and access tokens.

  • Client Certificate: An optional piece of data Skuid would present with all requests to positively, uniquely identify the sender of the request.

Using a REST data source

While REST models function very similarly to other DST models at runtime, configuring them properly requires an understanding of some unique concepts covered below.

Model Concepts

Methods

A REST model’s methods determine how the model can interact with the data it retrieves from the external system:

  • Query Method: Used to retrieve records
  • Insert Method: Used to create records
  • Update Method: Used to update existing records
  • Delete Method: Used to delete existing records

Some of these model methods can use different HTTP verbs for their operations. For example, query model methods may use GET or POST, while update model methods can use PUT, PATCH, or POST. Use your external system’s API documentation to determine which verb to select for each model method.

Data source URLs

Data source URLs are appended to the base URL within the data source’s configuration screen.

So a data source with a base URL of https://www.example.com/api/2/ may have data source URLs like the following:

  • contacts/
  • accounts/
  • opportunities/

The base URL ( https://www.example.com/api/2/ in the example above) should be consistent for all of the API’s endpoints and is configured in the data source. This base URL can vary in length from service to service, but typically includes the protocol (https:// in the example above), domain name (www.example.com/), and a URL path to related to the API being accessed (api/2)

The last portion of the URL will be configured at the model-level, for each individual method. Skuid automatically concatenates these two values to form a full URL, so there is no need to input the base URL from the data within each method’s data source URL.

Within these data source URLs, you may indicate sections of the URL as merge parameters using curly brackets. This will allow the creation of conditions, used to update the value of those URL merge parameters. Data source URLs are also compatible with global merge variables.

For example, the external system may expect the user to provide an account ID as part of the URL. To create a model supporting that behavior, your data source URL may look like: accounts/{{id}}.

Conditions

Once a condition is created, Skuid admins must select an available URL merge parameter—which are written in the model’s data source URLs—to update with a condition value.

Below the model, click URL Merge Conditions, and then click the Add Button. The parameter name automatically fills in with your merge variable. From there, set the condition as you would normally.

Response Formats

JSON and XML are two of the most common formats for receiving responses, but their structure varies significantly. Look at the following examples:

JSON

{
  "account": {
    "accountId": "8629c7899fa670c61126eee45b6d8826",
    "accountName": "Primary Account",
  }
}

XML

<account>
  <accountId>8629c7899fa670c61126eee45b6d8826</accountId>
  <accountName>Primary Account</accountName>
</account>

Both responses convey the same data, but their structure is very different. And note that, for XML-based APIs, even their structure may vary from the example listed above.

In general, Skuid has greater support for JSON-based responses, but XML APIs are also supported.

Because of the structure of XML, it’s possible to have attributes on a node in addition to its content. These attributes can be accessed when adding fields to the model by clicking the $ object when adding fields to the model.

So if a data source returned a response like this:

<account type="Partner">
  <accountId>8629c7899fa670c61126eee45b6d8826</accountId>
  <accountName>Primary Account</accountName>
</account>

You could access the type attribute by adding account > $ > type within the model’s Fields sub-tab.

Warning

Depending on the structure of some XML-based APIs, Skuid may not be able to interpret their responses correctly. Verify the structure of the API’s responses. If problems persist, consider using a snippet to intercept and translate the payload.

Note

When working with XML-based APIs, you may need to use custom payload templates, or (in some cases) snippets to interface with your chosen API.

Custom Requests / Payloads

All data sources function through HTTP requests and responses. Other DSTs abstract the logic of those responses and requests, but it is critical to be familiar with your service’s expected request/response format when using the REST DST. And while Skuid’s basic request options can work with a variety of systems, you may need to alter the way Skuid sends or receives data.

To do so, utilize the REST model’s templated requests, payload generation snippets, or payload intercept snippets. To visualize where and how these options affect requests, see the chart below:

rest-0

Request bodies

Every REST API is different, and sometimes services expect requests for certain operations to be formatted in particular ways. To facilitate the variety of ways different external systems may expect to receive data, several methods have a different options for formatting how data is sent. Several methods listed below may have these request body options available.

  • As JSON in request body: Skuid sends the updated fields as JSON in the request payload. For example, a complete payload could look like this:

    {"a-field": "This is the new value of the field.","another-field": "Also a new value."}
    
  • As nested JSON in request body: Skuid nests the updated fields object within a key specified in the Location of nested changes property. For example, a complete payload, nested in data, could look like this:

    {"data": {"a-field": "This is the new value of the field.","another-field": "Also a new value."}}
    
  • As URL-encoded Form data in request body: Skuid sends the updated fields—separated by ampersands (&)—as URL-encoded form data. These requests have a MIME type of application/x-www-form-urlencoded.

    Mainly used for APIs that expect form elements and POST requests. For example, a complete payload may look like this

    a-field=This+is+the+new+value+of+the+field.&another-field=Also+a+new+value.

  • In request headers - one per changed field: Instead of sending values within a payload, Skuid sends updated fields as additional HTTP headers for the request.

    So, in addition to standard request headers (like Content-Type and Origin), Skuid sends additional request headers on the request:

    Content-Type: application/JSON
    
    Origin: https://example.skuidsite.com
    
    a-field: This is the new value of the field.
    
    another-field: Also a new value.
    
    • Field name to header name mapping: Determines how field names are formatted for HTTP request headers.
      • Same names: If selected, headers will be labeled with their field names as selected from the model’s metadata.
      • Same names, with added prefix: If selected, parameters will prefixed with the value in the Headerult name prefix property.
  • In URL parameters - one per changed field: Updated field values are appended as percent-encoded parameters at the end of the method’s data source URL.

    As an example:

    A data source whose API endpoint: https://example.com/api/

    On a model with an update method pointing to 2/records/{{id}}

    Updating a record would result in a request to this endpoint URL:

    https://example.com/api/2/records/{{id}}?a-field=This%20is%20a%20new%20value%20of%20the%20field.&another-field=Also%20a%20new%20value

    • Field name to parameter name mapping: Determines how field names are formatted for URL parameters.
      • Same names: If selected, parameters will be appended to the data source URL with their field names as selected from the model’s metadata.
      • Same names, with added prefix: If selected, parameters will prefixed with the value in the Parameter name prefix property.
  • As custom payload format in request body: Sends whatever payload is returned by the selected payload generation snippet. Updated values are passed into the snippet as argument[0].changes. For more information, see the payload generation snippet section.

    • Payload Generation Snippet: The JavaScript snippet used to form and return the payload.
    • Payload Content Type: Determines the Content-Type of the HTTP request. Defaults to application/json.
  • As templated request body: Sends the payload as written in the Custom Request field that appears when this option is selected. Compatible with merge syntax. For more information, see the merge template custom payload section.

Sending custom requests

Requests can be customized for methods that use PUT, POST, PATCH, and DELETE HTTP verbs. As of Skuid release 12, this means query and delete methods may also utilize these custom payloads if they have the appropriate HTTP verb.

Note

If a custom payload is used, or if the response is intercepted by a snippet, Skuid will not attempt to query a model’s object for metadata. Instead, you must manually add any fields. Ensure you are familiar with the payload response your external system will send. This also means you will not be able to use Path to Contents option. If receiving a response where you must use Path To Contents, consider using an Intercept Respond Snippet to transform the response so that it has all data on the same object level.

Merge templates

When As templated request body is selected, admins may create their payloads within a template field on the method called Custom Request. This field is compatible with merge syntax, and allows for creating custom payloads quicker and easier than using JavaScript through custom payload snippets.

This option is particularly useful when API services require a different payload structure that is more complicated than just nesting changes within a JSON object.

For example, the following templated request:

{
  "changes": {
    "id": "{{id}}",
    "anArbitrarilyNestedValue": {
      "status": "{{status}}"
    },
    "description": "{{description}}",
    "arbitraryHardcodedValue": 1
  }
}

Would send a payload similar to the following:

{
  "changes": {
    "id": "8961403c236fd818162c5b9aa97b66ce",
    "anArbitrarilyNestedValue": {
      "status": "Okay"
    },
    "description": "This is the description of the record, pulled in through merge syntax.",
    "arbitraryHardcodedValue": 1
  }
}

You could also apply this same practice to an XML-based payload:

<changes>
  <id>{{id}}</id>
  <arbitraryNestedValue>
    <status>{{status}}</status>
  </arbitraryNestedValue>
  <description>{{description}}</description>
  <arbitraryHardcodedValue>1</arbitraryHardcodedValue>
</changes>

For more complex payloads—for example, those requiring additional data or string manipulation—consider using payload generation snippets.

Payload generation snippets

Payload generation snippets are used to alter the payload a Skuid model will send before the network request goes to the external system.

As with other Skuid snippets, you can use parameters passed in through arguments[0]. For payload generation snippets these include:

  • changes: An object containing key-value pairs of the fields being updated
  • model: The skuid.model.Model object representing the model being updated
  • row: The row object representing the row being updated

Using these parameters, you can craft custom responses in a format best suited to your data source. The most important part of payload generation snippets, however, is using the return keyword to return the generated payload to the Skuid model.

For example, it’s possible to emulate the As JSON in Request Body option by using this snippet:

var params = arguments[0],
  $ = skuid.$;
return params.changes

Since payload generation snippets allow for the use of other web APIs and Skuid APIs, they are most useful for very complex payload structures that may not be possible through a merge template, or that require some additional manipulation to achieve the desired effect.

Intercepting response payloads

Some external systems may return payloads in a format that Skuid cannot parse. If network requests are succeeding and returning responses, but Skuid components are not utilizing the data properly, you may need to intercept and alter the response.

As with other Skuid snippets, utilize parameters passed in through arguments[0]. For snippets that intercept responses these include:

  • response: The full response returned by the external system with several nested fields:

    • body: The full response body returned by the external system

    Note

    Whether or not this is a JSON object or XML document depends on the API being accessed.

As with payload generation snippets, Skuid expects these snippets to return data in a JSON object that Skuid recognizes. Skuid expects that each model field will be a separate key in the returned object. If the method has a Path to Contents value, then the fields must be nested within that key as well. Unless data is returned, the model’s data will be unusable by Skuid components.

Models Properties

In addition to several standard model properties, REST models have some unique properties as well:

  • Model behavior: Determines whether a model will be used for exclusively for querying data, or for querying and updating data.
    • Read-only: Designates that the model can only query. Since only one method will be available, the query method’s properties will be listed on the model itself.
    • Read/Write: Designates that the model may use multiple data source URLs for different data operations.
  • Model Label and Model Label Plural: The labels used to represent object being accessed by the model, since that metadata may not be available. These labels can be seen in Skuid’s component text, such as Add New <Model Label> or Show 10 <Model Label Plural> Per page.

Method Properties

Query

  • Data source URL: The URL route—which is appended to the data source’s base URL—to send requests to for this method. Using merge syntax in property will create URL merge variables for use with model conditions. See the data source URLs section for more conceptual information.
  • Data Source HTTP Verb: The HTTP verb to be used. Available options for this method include:
  • Intercept Response Payload: A true/false property indicating whether or not any responses received from this method should be intercepted and processed by a JavaScript snippet.

  • Path to Contents: Specifies the model’s starting point within the payload that’s returned from the service. For example, an external system may consistently return a record data nested within a JSON key:

    {
     "records" : [
       {
         "Id" : 1,
         "Name" : "Leeroy Jenkins"
       }
     ]
    }
    

    In this case, the Path to Contents would be set to records, as the data needed is always within that key. Since some Skuid features may not function as intended with nested fields, it’s typically best to choose a Path To Contents.

    Note

    Using the Path to Contents property is typically not recommended with XML-based APIs.

  • Response Format: Determines whether Skuid should expect responses formatted as JSON or XML. Must match the format returned by the API or could cause errors. For more information see the Response Formats section.

  • Limit Method: Determines whether or not the model sends a query containing a URL parameter that limits returned records.

    • Limit Parameter: The parameter to be appended to the HTTP request that determines the amount of records requested.
    • Max # of records (limit): The maximum number of records to be returned in a query.
  • Pagination Method: Determines whether or not the model sends a query containing a URL parameter that defines how returned records are returned.

    • Offset Parameter: The parameter to be appended to the HTTP request that determines how many records are skipped in the load more request. For example: /someapi/gimmecats?limit=5&offset=5 - five cats, but the second group of five cats.

Insert

  • Data source URL: The URL route—which is appended to the data source’s base URL—to send requests to for this method. Using merge syntax in property will create URL merge variables for use with model conditions. See the data source URLs section for more conceptual information.
  • Data Source HTTP Verb: The HTTP verb to be used. Only the POST verb is available for this method.
  • Method Succeeds If…: Determines what Skuid considers a success state for this method.

    • Request succeeds: Successful if the network request returns a 200 OK status response

    • “Success” Field in response body is present: Successful if the field noted in this property’s sub-property is present

      • Success Field Name: Determines which field label Skuid will consider as the success field.
    • “Success” Field in response body is true: Successful if the field the chosen success field has a value of true.

      • Success Field Name: Determines which field label Skuid will consider as the success field.
    • Response body equals a value: Successful if the entire response body equals the value of this property’s sub-property

      • Contents of response body if request succeeds: The value Skuid will compare to in order to determine success
    • “Success” Header is present in response: Successful if the header noted in this property’s sub-property is present

      • Success Header Name: Determines which field label Skuid will consider as the success field.
  • Response handling: Determines how Skuid should expect the external system to confirm a new record has been created.

    • Created record will be returned in the response body

    • ID of created record will be returned in field in response body

      • Name of Field containing created record’s Id
    • ID of created record will be returned in response header

      • Name of Header containing created record’s Id
    • No information about the created record will be returned

Update

  • Data source URL: The URL route—which is appended to the data source’s base URL—to send requests to for this method. Using merge syntax in property will create URL merge variables for use with model conditions. See the data source URLs section for more conceptual information.
  • Data Source HTTP Verb: The HTTP verb to be used. Available options for this method include:
  • Intercept Response Payload: A true/false property indicating whether or not any responses received from this method should be intercepted and processed by a JavaScript snippet.
  • URL expects batch updates: If true, Skuid will send information about any affected rows as a single network request. If false, Skuid will send separate requests for every row that is affected by an operation.
  • Send updated field values…: Determines how Skuid will format the request body for this method. See the request bodies section for more information on each option.

Delete

  • Data source URL: The URL route—which is appended to the data source’s base URL—to send requests to for this method. Using merge syntax in property will create URL merge variables for use with model conditions. See the data source URLs section for more conceptual information.
  • Data Source HTTP Verb: The HTTP verb to be used. Only the DELETE verb is available for this method.
  • Method Succeeds If…: Determines what Skuid considers a success state for this method.

    • Request succeeds: Successful if the network request returns a 200 OK status response

    • “Success” Field in response body is present: Successful if the field noted in this property’s sub-property is present

      • Success Field Name: Determines which field label Skuid will consider as the success field.
    • “Success” Field in response body is true: Successful if the field the chosen success field has a value of true.

      • Success Field Name: Determines which field label Skuid will consider as the success field.
    • Response body equals a value: Successful if the entire response body equals the value of this property’s sub-property

      • Contents of response body if request succeeds: The value Skuid will compare to in order to determine success
    • “Success” Header is present in response: Successful if the header noted in this property’s sub-property is present

      • Success Header Name: Determines which field label Skuid will consider as the success field.
  • URL expects batch updates: If true, Skuid will send information about any affected rows as a single network request. If false, Skuid will send separate requests for every row that is affected by an operation.

  • Request body contents: Determines how Skuid will format the request body for this method. The following options are available:

    • None
    • As JSON in request body
    • As custom payload format in request body
    • As templated request body

    See the request bodies section for more information

Troubleshooting

Numbers appear in model’s Fields screen [[]]

You may be receiving an XML response while the model is configured to expect JSON payloads. Consider setting the Response Format property to XML.

Depending on the API, you may also be able to indicate you want to receive JSON responses by adding this to your data source’s Common Request Headers:

Accept: application/json

Also consider updating the query method’s Response Format property to XML.

Skuid hangs when loading model metadata [[]]

The external system may not support the data source’s chosen grant type. For example, if using the Client Credentials grant type, try using the Authorization Code type instead. Verify which grant type to use within your API’s documentation.