JMESPath

What is it?

JMESPath is a query language for JSON. WIth it you can script searching large JSON datasets and transform the results.

Basic queries

A search can be as simple as a single field name. result would return the object attached to the result field at the top level of a JSON file. This can be specialized to return an object inside result by added a dot and then another field name eg result.object.

Queries can index arrays as follows

users[0].name

Which will return the name field of the first element of the array users. You can use a negative index to access an array from the end, so the index of -1 will give you the last element of an array.

Projection queries

Projections allow you to apply a query to multiple objects. The left hand side of a projection query defines the projection and the right hand side defines the query to apply to each object returned by the projection. Using the example from above by replacing the array index with a * we create a list projection users[*] which we then apply the query name to. The following returns the name field from every element in the users array.

users[*].name

There are five different types of projection:

• list: applies query to each object in array as demonstrated above.
• slice: applies query to each object in a slice from an array
• flatten: applies query to each object in an array and flattens the result into a parent array
• object: applies query to all the child objects
• filter: applies query to a filtered version of the object

As this article isn't meant to cover the full spec of JMESPath but just give you a feel for what can be done with it I'll only cover filter projections here. A filter projection allows you to filter the left hand side of the projection before the query on the right hand side is applied to it. For example the following will return all the entries in the users array that have the field country set to USA.

users[?country=="USA"]

Functions

JMESPath has a number of standard functions that can be applied to results within a query. Using the same users example we have been using throughout we can sort the results by placing it inside the sort function.

sort(users[*].name)

You can also use functions inside filter projections. The following would return all the users whose names start with A.

users[?starts_with(name, "A")]

Transforming the results

If you want the results of your query in a specific format you can use multi-select queries. These can be used to create arrays or objects from your results. If I'm only interested in the name and age fields of the entries in my users array I can transform the list query results using a multi-select hash

users[*].{"name": name, "age": age}

Why am I telling you this

Well, AWS uses JMESPath in their SDK generation. If you have used their CLI tool you might recognise the syntax from the --query option. So far in my work on Soto (a third party Swift SDK for AWS) I have been able to hardcode the queries I came across and write custom code for them. Through recent work though, I have found a number of queries that were too complex. So I took the leap and decided to implement JMESPath for Swift. You can find the code here. Given there are versions of JMESPath for a number of languages already I had a lot of prior art to help in my implementation. The Swift version is based off the Rust version and follows its implementation fairly closely. The JMESPath site provides a comprehensive set of compliance tests and my version passes them all so I am fairly confident it is up to scratch.

The Swift code works as follows. You create a compiled Expression and then use Expression.search to apply the expression to your JSON.

let expression = try JMESExpression.compile("users[*].name")
let result = try expression.search(json: myjson)

The above will return Any as it is undefined what type the expression will return. You can pass this straight into JSONSerialization.data to get the result as JSON if desired. Or if you know the type that the search will return you can add it into the search call as follows

let result = try expression.search(json: myjson, as: [String].self)

In addition to parsing JSON I've added Mirror reflection to search objects of any type. This means the Swift implementation of JMESPath isn't restricted to just querying JSON data it can be used to query any data structure. Below is a sample where we have a struct Service which includes an array of User. We use JMESPath to create a sorted list of all the User member name.

struct Service {
    struct User {
        let name: String
    }
    let users: [User]
}
let service = Service(
    users: [
        .init(name: "james"),
        .init(name: "anne"),
        .init(name: "tom")
    ]
)
let expression = try JMESExpression.compile("users[*].name | sort(@)")
let result = try expression.search(object: service, as: [String.self])
assert(result == ["anne", "james", "tom"])

The expression used here includes two new pieces of grammar | which pipes the result of the expression on the left hand side into the expression on the right hand side and @ which indicates the current result being evaluated. So above the array of names returned by users[*].name is passed into the sort function.

Usage

JMESPath provides an easy way to script filtering and transforming of JSON responses from different services. It can be used to parse responses from cloud providers and other services from your backend into the formats that your client expects.

More Information

This article only details a small sample of the possibilities with JMESPath. If you'd like to find out more about JMESPath you can go the website jmespath.org. It has a great tutorial section where you can test out different queries.

You can checkout the Swift version of JMESPath here https://github.com/adam-fowler/jmespath.swift.