The Basics with Node.js Examples

Introduction

Public-facing APIs have tremendously increased in the last couple of years. Businesses have seen that sharing their business data with the public can be beneficial. There are many reasons for this: such as the fact that it allows the API users to create something new and interesting with the shared data, and that APIs enable the creation of communities around a particular business. Take Twitter. You can perform almost any action on it without being actually on Twitter. Such public APIs are mostly managed and not naked, meaning that they control API requests, impose a rate limit and implement other measures of control such as requiring an access key and allowing only certain IPs to use the API with a given key. Below, we will discuss two simple ways to implement authentication in Web APIs and exemplify those authentication methods with Node.js and WordPress. We will present an introduction on working with CORS in Node.js, JSONP and we will introduce some API conventions.

Figure 1: A small part of teh Twitter's API endpoints

Feel free to download all the associated code examples with this article.

[download]

They are located in an .html file, are nicely formatted, and ready to be pasted in a code editor for you to try out and use.

Figure 2: A part of the page with the code in the downloadable files associated with the article.

Authentication

If you are not going to make an open API that can be accessed anonymously, you would need to control access and ensure the information's CIA is untampered. Otherwise, you just need to worry about rate limits. If you use the HTTP Basic Authentication where you must send a 401 Not Authorized response and await an Authorization header in the following user's request, you would have to worry about the confidentiality of the credentials. They can be leaked both while in transit between the client/server and while being at rest. You would need to implement an encrypted channel (HTTPS - HTTP over TLS) to make sure the credentials the user provided would not be leaked while traveling back and forth and encrypt them (using one-way hashing and salts) while the user's data is in rest. An example of an API that uses the HTTP Basic Authentication is the GitHub's API: see https://developer.github.com/v3/auth/, Delicious' API and many more. An example of setting up a HTTP Auth page in Node.js could be seen in

Excerpt 1 in the download files (code1.html).

We have hardcoded the credentials for simplicity's purposes, but traditionally the passwords would be kept in a file such as .htpasswd (Apache). There is such functionality available in Node through a module as well: https://www.npmjs.com/package/htpasswd. Note that there are modules available to do the heavy lifting for HTTP Auth and Digest such as http-auth

Consuming a HTTP Auth Web API

1. WordPress

   If you are planning on consuming a Web API that uses HTTP Auth,

   Excerpt 2 shows how you can do that within the WordPress API. In the example, we consume the Delicious API to get the links of a given user and optionally – only the links for a particular tag. You have to define the $username , $password and set the $tag variable to any string or to an empty string if you want all tags.The resulting $body variable will hold the returned page from the AJAX call.

2. Core Node.js

   To use a Web API with HTTP auth in core Node, we use the get method of the http module, pass it an options object with host/path and the authentication header (we do not have to encode with base64 the username and password) and process the response. You can see a way to do it in

3. Excerpt 3 where we consume the Delicious' API.

HTTP Digest Authentication

The major difference between this and HTTP Basic Authentication in terms of procedure is that the client must first make a request to the server with no credentials. Then, the server should respond to this request with a nonce, which is a unique data string generated by the server whenever there is a 401 Not Authorized response, and a realm (which pinpoints what part of the website the user is accessing and therefore give a clue as to which credentials he must provide as there may be multiple realms in a website requiring different credentials). Finally, the client responds providing the nonce and his encrypted credentials - username, password and realm and the server either delivers him the requested information or throws an error. To set up a HTTP Digest Authentication you can use the digest module (npm install digest) You create a server through the returned object from requiring digest, pass it your username and password followed by the callback that will be your response body for authenticated users. You can see how we set up a page with digest authentication in Node.js (with the digest module) in

Excerpt 4.

Consuming a Web API that uses Digest

You could create your own module to handle that like we built for the HTTP Auth or use what is already out there - something like the http digest client

HTTP Basic Authentication or HTTP Digest Authentication?

This decision is ultimately at your discretion but here are some factors to consider:

HTTP Digest Authentication HTTP Basic Authentication

Credentials are never in plain text while in transit Credentials in transit are always in plain text

No other security systems necessary Require the addition of other security systems such as HTTPS to be secure

Either credentials should be stored in plain text, or the entire set of credentials should be hashed including the username and the realm. With data at rest - passwords can be kept in the form of a salted hash

Can also be configured to protect the integrity of the incoming data Can only be used for Authentication purposes

Every call required urges the client to make 2 requests Requires SSL to be secure which runs slower than basic HTTP

CORS (Cross-origin resource sharing)

If you want to build APIs that could be used only by a particular website, you should not allow cross-origin access to your endpoints. For example, if you have a website that allows users to create, read, update, and delete notes and a RESTful API, which manages these tasks you may not necessarily want people to add notes to your users list of notes while they are browsing a third-party website. You certainly do not want attackers to use the end user's logged-in status to access that user's notes and go behind the login wall. Below is a sample excerpt from a RESTful API in Node.js: It does not have a persistence layer and functionality for updating or deleting messages but it does offer the possibility to add messages. Those are just added in an array and persist until the server is restarted. If you plan not to let third parties use that API - it is fine as it is but some changes must be made if you want it public. You can see the sample RESTful API in

Excerpt 5.

You can see a POST request adds a new message successfully when we test it with cURL:

>> PS C:UsersIvan> curl --data "Ca va" --noproxy localhost, http://localhost:9999/message

>> Message: Ca va added successfully

A subsequent GET request to /message shows all messages

>> PS C:UsersIvan> curl --noproxy localhost, http://localhost:9999/message

>> Hello -Bonjour -Ni Hao -Zdravey -Ca va

A GET request to /message/#{id} shows that array index:

>> PS C:UsersIvan> curl --noproxy localhost, http://localhost:9999/message/3

>>Ni Hao

But what would happen if another website uses our browser to make an AJAX call to our API endpoints to carry out an action for the user? Excerpt 6 contains our second Apache server (which attempts to asynchronously insert a new message to our Node server).

You can see in the picture below what will happen if we attempt to asynchronously insert a new message to our other Node server.

Figure 3: The result when we open localhost:8079 in a recent browser

However, that protection would not work on older browsers leading to possible CSRF and other security issues if we do not purposefully enable CORS for AJAX calls. Now, if we are creating a public API we may want other websites to access our APIs endpoints and take some actions (at least some of them). CORS is disabled by default for AJAX requests and no other site would be able to make AJAX calls to your API endpoints. This might be desirable for this demo app but there are many cases when you explicitly want cross-domain AJAX calls. To enable all origins, you must add specific headers when sending back a response to the user – you can see them in

Excerpt 7. This will result in our second server (the Apache) returning the following alert:

Alternatively, you may want to enable only particular domains from accessing the API: you just replace the wildcard * with the particular domain names you accept: res.setHeader('Access-Control-Allow-Origin', 'http://example.com');

JSONP

An alternative to enabling CORS is returning json and allowing your API users to use the JSONP technique. Because AJAX would not be possible when requesting data from different domains without the request being specifically allowed you can call the API using a <script> tag which is not prohibited by the same-origin policy. Typically, the API would return the data to a function in your code defined in a GET argument like ?callback= that you set when you were embedding the external script. A notable drawback is that with JSONP you can only perform GET requests. If you are using jQuery you can perform a JSONP call by simply adding dataType: 'jsonp' to your options object for $.ajax(). An example of using JSONP in jQuery is located in

Excerpt 8. You can see that jQuery is doing the job behind the scenes. Here is how it usually works without jQuery: Firstly, you add a script pointing to the API you will be consuming, specifying a function in your code which will be called. <script src="http://some-site-here.com/api.json?callback=someSiteData"></script>

You actually have to create the function which would accept the data and process that data in any way you want which could be something like function someSiteData(data) { alert(data); }

Excerpt 9 contains an example in which we use the GitHub's Status API and alert the status of the website and the time when this status was last updated.

APIs Conventions

1. Use a different domain or sub-domain for the API endpoints. (Facilitates migration and scaling later on)

2. Include the version number of the API in the URL (this would allow a step-by-step deprecation of an API before it is retired)

3. Include the particular endpoint (resource) after steps 1 and 2. Here is an example: https://api.linkedin.com/v1/people/~ The LinkedIn API to fetch people uses a subdomain (api.linkedin.com), then the version number (/v1/) is in the path to the resource and lastly the endpoint includes the resource that is going to be fetched. RESTful resources should have a structure resembling the one below. In this example, we have used a resource called message but it could be any resource. Also, we have used the syntax #{id} to refer to a random integer or string identifying a particular resource.

1. GET request to /message -> returns all messages in the database.

2. GET request to*/message/#{id}*-> returns the message that matches a given id

3. POST request to /message -> creates a new message in the db.

4. PUT request to /message/#{id} -> Updates the message with the given id.

5. DELETE request to /message/{#id} -> Deletes the message with the given id from the db.

When considering whether to use XML or JSON for data interchange in your API it must be noted that JSON is the de facto standard nowadays. JSON is more efficient in terms of performance - the data is represented as JavaScript objects that are widely used by JavaScript and other languages, leads to fewer bits in transit and requires less machine time for processing. XML is great for structured information with the added benefit that the application's logic responsible for processing the incoming data would not have to validate it on its own because of the XML Schema Definition which already defines how the XML document has to be structured (The Schema could define all legal building blocks of the document). Another great thing about it is that it is highly human-readable. Nonetheless, all those tags and attributes add a lot of data payload, which is not desirable in contemporary web services that have to be fast. Here is an example of how you can save from the data flowing through the wire using JSON (albeit the XML used consists only of elements and is inefficient purposefully)

Excerpt 10 shows some XML data concerning books.

Excerpt 11 shows the same data but conveyed using JSON. The XML variant contains 548 characters while the JSON one contains 222 characters.

If you want a more rigid data structure then XML is the way to go for now. However, if performance is the primary factor - JSON is the way to go.

Conclusion

There is much more to know about APIs and about their security. We have barely scratched the surface here but I hope this can serve as a healthy introduction to the world of APIs. I will continue this topic with some more advanced concepts including JWT, OAuth and the Internet of Things. And you, are you currently working on any public APIs and what mechanisms for security and authorization are you planning to use?

References

1.Wikipedia, 'Basic Access Authentication', Available at: http://en.wikipedia.org/wiki/Basic_access_authentication

2.Mark Kirby, 'How to authenticate APIs – HTTP Basic vs HTTP Digest', Available at: http://mark-kirby.co.uk/2013/how-to-authenticate-apis-http-basic-vs-http-digest/

3. API Evangelist, "Considerations When Planning Endpoints for your RESTful API", Available at: http://apievangelist.com/2011/10/18/considerations-when-planning-endpoints-for-your-restful-api/

4.Sean Lindo, 'XML VS. JSON - A PRIMER', Available at: http://www.programmableweb.com/news/xml-vs.-json-primer/how-to/2013/11/07

5. Rob Zazueta, 'API Data Exchange: XML vs. JSON', Available at: http://www.mashery.com/blog/api-data-exchange-xml-vs-json

6. Wikipedia, 'Cross-origin resource sharing', Available at: http://en.wikipedia.org/wiki/Cross-origin_resource_sharing

11 courses, 8+ hours of training

Learn cybersecurity from Ted Harrington, the #1 best-selling author of "Hackable: How to Do Application Security Right."

See Courses

7.Wikipedia, 'JSONP', Available at: http://en.wikipedia.org/wiki/JSONP