14.6.18

Flow action request

The request action carries out a HTTP or HTTPS request to load content from remote sources.

Syntax

  • content="..." to set the id for a request (optional, default is main)
  • url="..." to set the request URL (required)
  • method="GET|POST|..."to set the request method (optional, default is GET)

The request element may contain any request options.

Examples:

<request content="main" url="http://example.com">
  <timeout request="1" connect="2" />
  <header name="X-A" value="A" />
  <query name="y" value="b" />
  <post name="x" value="a" />
  <upload name="fieldName" filename="path_to_file.txt" size="123" type="text/plain" src="fit://request/uploads/myFile" />
</request>
<request content="weather" url="http://example.com/weather" />

Usage

The response body will be written into fit://request/content/<ID>. Additional information about the response, such as headers and status code can be found in fit://request/content/<ID>/response.

The main request (with content="main") receives a special treatment. See default-request action for more information. Usually you should use default-request to obtain the main content.

If you want to request more than one content, you should use the requests action that is able to execute requests in parallel. The syntax is the same. Thus wrapping a <request> with a <requests> parent element is equivalent.

All request options set in conf/sources.xml file that match the requested domain and path will also be applied to your request. However, options in the action have precedence over those from conf/sources.xml.

Be careful with selection values for the name attributes in input elements (<input name="" ... />) in your HTML documents. PHP automatically replaces many signs to underscores (_), so it can lead to unexpected field names in your backend systems. In some cases PHP removes even the fields completely from request.

Examples:

<input name="test.abc" /> <!-- Result: "test_abc" -->
<input name="test.abc[]" /> <!-- Result: "test_abc[]" -->
<input name="test[abc].xyz" type="input" /> <!-- Result: "test[abc]" -->
<input name="test[abc].xyz" type="file" /> <!-- Result: this field will be completely removed -->

Raw POST Requests

Simple POST requests can be created as follows. Here we send a POST request to the httpbin Request & Response Service:

<flow>
  <request url="https://httpbin.org/post" method="post" />
  <dump/>
</flow>

To get a working example, we also have to allow the origin in the ACLs:

<allow url="https://httpbin.org:443/" />

For convenience, our example flow simply dumps the supplied JSON response wherein https://httpbin.org/ just reflects the request we’ve just sent:

{
  "args": {},
  "data": "",
  "files": {},
  "form": {},
  "headers": {
    "Accept": "*/*",
    "Accept-Encoding": "deflate, gzip",
    "Connection": "close",
    "Content-Length": "0",
    "Host": "httpbin.org",
    "User-Agent": "Sevenval FIT"
  },
  "json": null,
  "origin": "94.216.65.254",
  "url": "https://httpbin.org/post"
}

Let’s now send some data in the body field. Since the HTTP method will be automatically set to POST, we can omit method="post" here:

<request url="https://httpbin.org/post">
  <body>foo=42&amp;qux</body>
</request>

The response shows that FIT sends the data as (application/x-www-form-urlencoded), i.e. as URL-encoded form data foo=42&qux:

{
  …
  "form": {
    "foo": "42"
    "qux": "",
  },
  "headers": {
    …
    "Content-Length": "10",
    "Content-Type": "application/x-www-form-urlencoded",
    …
  },
  …
}

By the way, have you noticed how we XML-encoded the ampersand as &amp; in the request? Not only do we have to consider the XML context we’re in, but we also need to URL-encode the data ourselves, as the body field contains the raw POST data as they will be sent to the origin.

The post field to the rescue! We can achieve the same result with the following, much simpler request, automatic URL-encoding included:

<request url="https://httpbin.org/post">
  <post name="foo" value="42" />
  <post name="qux" />
</request>

Thus, for submitting URL-encoded form data, the post field is the way to go.

Let’s go back to the body field. As we’ve seen before FIT automatically sends the data as application/x-www-form-urlencoded. We can change that by adding a different Content-Type header field:

<request url="https://httpbin.org/post">
  <header name="Content-Type" value="text/plain" />
  <body>foo</body>
</request>

{
  "args": {},
  "data": "foo",
  "files": {},
  "form": {},
  "headers": {
    …
    "Content-Length": "3",
    "Content-Type": "text/plain",
    …
  },
  …
}

Next, we will send an XML body. First, we set the MIME type in the header field to text/xml and supply the XML payload as text (!) in the body field:

<request url="https://httpbin.org/post">
  <header name="Content-Type" value="text/xml" />
  <body><![CDATA[<foo/>]]></body>
</request>

Instead of using <![CDATA[…]]> we could have written <body>&lt;foo/></body>. In contrast, <body><foo/></body> will not produce the desired result! (cf. the mode=xml documented below).

{
  …
  "data": "<foo/>",
  …
  "headers": {
    …
    "Content-Length": "6",
    "Content-Type": "text/xml",
    …
  },
  …
}

The body does not have to be placed inline. We can also make use of the body field’s src attribute to specify a location where the body content should be read from:

<request url="https://httpbin.org/post">
  <header name="Content-Type" value="text/plain" />
  <body src="fit://site/conf/foo" />
</request>

This way is binary safe, so we could also send an image or any other content:

<request url="https://httpbin.org/post">
  <header name="Content-Type" value="image/jpeg" />
  <body src="fit://site/public/foo.jpg" />
</request>

{
  …
  "data": "data:application/octet-stream;base64,/9j/4AAQSkZJR…
  …
  "headers": {
    …
    "Content-Length": "15198",
    "Content-Type": "image/jpeg",
    …
  },
  …
}

The referenced content may be created dynamically with XSLT:

<xslt src="body.xsl" out="fit://request/dynamic-body" />
<request url="https://httpbin.org/post">
  <header name="Content-Type" value="text/xml" />
  <body src="fit://request/dynamic-body" />
</request>

Posting JSON

There is a special mode for JSON request bodies. You may you use the JSON XML notation inside the <body> element:

<request url="https://httpbin.org/post">
  <body mode="json">
    <json-document>
      <cool type="boolean" value="true" />
    </json-document>
  </body>
</request>

With mode=json, the body will be converted into a JSON document:

{"cool":true}

This mode also works with body content sourced from a fit:// location.

<request url="https://httpbin.org/post">
  <body mode="json" src="fit://request/some-json-xml-body" />
</request>

If no Content-Type header was set explicitly, application/json will be used. As usual, the Content-Length is calculated automatically. For all requests with body, the default HTTP method is POST. You may override that, e.g. with PUT.

Posting XML

Sending an XML request body is a common task, too. The xml mode let’s you write inline XML inside your request definition:

<request url="https://httpbin.org/post">
  <body mode="xml">
    <message>foo bar</message>
  </body>
</request>

In contrast to the raw POSTs (mode=text) that we have seen above, in XML mode the POSTed body is

<message>foo bar</message>

If not set explicitly, a Content-Type: text/xml header will be added.

Note, that <body> may only contain one child element. That child will be XML serialized with all its children. This includes XML comments, too.

When you are working with XML namespaces, you have to repeat the namespace definition inside the request’s <body> element.