Why we decided to try eandata.com as a barcode lookup provider for unwaste.io - and how to call the eandata.com/feed API.
Years: 2020.
Working on unwaste.io, one of the challenges is to help waste spotters quickly input the specific type and brand of discarded plastic bottles. This is not always easy on the small screen of a mobile app, especially if you were in the middle of something else and just stopped for a moment to log a bottle. The obvious solution was to use the bottle's barcode. Scanning the barcode itself is a different challenge, which may be the subject of a different post - but it is certainly doable.
Perhaps surprisingly, matching that barcode to useful product information is much more difficult than it might first appear.
A first option is to use the national representatives of the GS1 organization which oversees the distribution of the company codes (GTINs) which are embedded in most barcodes. This option was not ideal for several reasons: poor coverage in the countries unwaste.io is focused on; high fixed fees; the need to deal with multiple organizations.
A second option is to use one of several commercial API services (such as Barcode Spider ) aimed at providing "enriched" information about products sold on major ecommerce platforms, particularly Amazon. In the case of Unwaste, these were not so relevant as they do not contain the kind of consumer product, widely sold in developing countries, which Unwaste is concerned with. Furthermore they can be quite pricey.
A final option is "freemium" providers who typically provide a free/cheap basic tier, and a supported/higher volume paid tier, plus the ability for submitting new data. As Unwaste aims to crowdsource its information, and also to contribute data to other crowdsourced providers, this option seemed ideal.
I therefore spent several days searching for, and trying out, several providers. Below is what I found, hopefully it can save other readers some time:
http://upcdatabase.org/ : has a lookup and insertion API, well-documented, and seems to be active. However it has no support. In fact it appears that the site maintainer has purposefully removed any and all contact methods for interaction. Upon testing I came across a few issues with the API, and this plus the lack of support meant it was not a viable option for me.
https://www.barcodable.com/ : well documented, with a free and paid tier, and clearly-stated support option. However no API for inserting new data - a pity, otherwise this would have been ideal for our use case.
https://www.buycott.com/api : no free tier, but several paid tiers with clear support options. Unfortunately the documentation and feature set is not available without signing up to a paid plan, so I did not investigate this further.
https://barcode.monster/ : only free tier, however some ads in slightly confusing places, documentation is rather patchy, no API for inserting new data.
https://eandata.com/ : free and paid tiers, documentation is pretty comprehensive however organized in a "blog" fashion which makes is quite difficult to use. Also has an interesting approach whereby inserting new data, buys you "credits" which can be used to pay for lookups. The site maintainer is active and responded helpfully to my support queries.
Based on the above, I decided to use eandata.com . The main documentation for the API is available here, however as it is spread over several posts, I have summarized below the key points from the point of view of using the API with REST+JSON.
The first point to note is that it is aimed at server-side applications. In the words of its maintainer:
the real intent of the API is for your server to make the call directly to my server rather than every client making the call. This allows your server to cache the results for future use and manipulate the format as you see fit.
One consequence of this is that the API does not support CORS-handling, meaning that calling it from a CORS-enforcing browser will require going through a suitably-configured proxy.
The eandata API endpoint for both product lookup and insert is: https://eandata.com/feed/ .
Note that the trailing forward-slash is essential.
In this tutorial I will only be discussing version 3 of the API, used with JSON, therefore in the query string I will always include two parameters to indicate that, as follows:
https://eandata.com/feed/?v=3&mode=json
In order to use the API you will need an API Key, you can obtain one by registering here and then visiting https://eandata.com/feed/ in your browser.
The APIKey should be included in your request URL with the keycode parameter as follows:
https://eandata.com/feed/?v=3&mode=json&keycode=BF9XYZXYZXYZZZZZ
N.B. the keycode shown above is fake, you should replace it with your own keycode.
Once you have the above working, you can perform a lookup by using the find parameter to point to the product's barcode. For instance:
GET https://eandata.com/feed/?v=3&mode=json&keycode=BF9XYZXYZXYZZZZZ&find=6009832820507
The response will be a JSON-structure which is described in detail here.
I will just highlight some key aspects in parsing and interpreting the response.
status/code field in the JSON response.
This code generally follows HTTP RESTful principles, so 200 means a successful lookup, 404 no product found, 500 invalid barcode format, etc.product/attributesproduct/imagecompanyThe API allows for updating a single field at a time, or multiple fields at a time, or multiple fields of multiple products at a time. The unwaste use case is multiple fields of a single product so I describe how to do this below.
N.B. if you are testing the API with fake data, please ensure to set include a test=1 parameter in your query string, in order to avoid eandata's server permanently storing that data.
The basic approach will be the same as for a Product Lookup, with four important differences:
update=<barcode> rather than find=<barcode>field=*bulk* (yes, the asterisks must be included as shown)fields parameter in the QUERY STRING (not the HTTPRequest body).The format of the JSON structure to be submitted is (somewhat confusingly) different from the format of the lookup response structure. It is described here.
An example of this is shown below:
POST https://eandata.com/feed/?v=3&mode=json&test=1&keycode=BF9XYZXYZXYZZZZZ&update=6009646581007&field=*bulk*&fields=[{"field":"product", "value":"Vumba Mineral Water 1500ml"},{"field":"category", "value":"19"},{"field":"description", "value":"1.5 L"}]
The JSON response returned has the following key aspects:
status/code as for Product Lookupproducts/fields,
as an array of field:status pairs, where field is the fieldname you submitted and status should be "ok" if the field's value was accepted.Images require some additional explanation. Firstly, there is apparently a way to upload image data programmatically. It is described here, however I have not tested it.
Rather, I tested the alternative approach described here: https://eandata.com/blog/e107/data-feed-api-v3---handling-product-images/ . In this approach, your application does not upload the image /data/ but rather a URL from where the eandata server can pull the image data and store an appropiately-formatted version on its own servers. Therefore the URL provided should be accessible to eandata's server via a single GET request, without requiring any prior login, authentication cookie, or API key.
Secondly, the image URL should be submitted as a separate parameter in the query string, rather than being included in the fields parameter described above. An example is shown below:
https://eandata.com/feed/?v=3&mode=json&keycode=BF9XYZXYZXYZZZZZ&update=6009646581007&field=*bulk*&imageURL=http://www.aguavumba.co.mz/images/garrafa.png
If your update is successful, status/code should be "200" and products/img should be "ok".
If eandata.com could not process the submitted imageURL, products/img should contain an explanatory error message.