Subtitle: Why do in a day what you can spend two weeks automating?
Credit to Terrence Parr from whose motto this is derived.This post discusses creating a REST service to provide all or specific parts of the Avrdude configuration file as JSON.
Atmel produce a wide range of MCUs and these MCUs are a common component of many circuit designs. But before they can become useful in a given circuit they need to be programmed. For hobbyist projects this is commonly done using a bootloader. However for this the MCU needs to be preprogrammed with the bootloader itself.
An alternative that requires no preprogrammed bootloader is in-system programming (ISP). Typically the MCU is mounted along with all the other components on a given PCB and the PCB includes an ISP header which allows the MCU to be programmed directly on the board using a programmer.
The ISP header is generally a 6-pin header that just exposes a few pins of the MCU (reset and 3 SPI pins) along with ground and the positive power supply.
You can use an Arduino as a programmer or use a purpose built programmer like the USBtinyISP.
The programmer is mainly just hardware, you then need a utility that uses the programmer to e.g. actually upload a sketch to the flash ROM of the chip being programmed.
One of the best known tools for doing this is Avrdude (the Arduino IDE uses Avrdude under the covers).
To be able to do its work Avrdude needs to know various details about the programmer being used and the chip being programmed (the kind of details that can be found in the datasheet for a given MCU).
Avrdude currently knows the details for about 90 programmers and 150 MCUs (parts in Avrdude terminology).
Note: many of the programmers defined for Avrdude really just correspond to different configurations of particular given programmers.
The definitions for these programmers and parts are found in avrdude.conf, the source for which can be viewed in the Avrdude SVN repository as avrdude.conf.in.
For standalone projects where people want to be able to program just one MCU type or a limited number of MCU types, without requiring a computer capable of running Avrdude, the easiest thing is to just scrape the details for the relevant MCU or MCUs from avrdude.conf and hard code them into the logic that will be handling the programming.
E.g. see the code for the Adafruit standalone AVR chip programmer or Nick Gammon's ATmega board programmer (both of which are based on or inspired by Bill Westfield's OptiLoader).
The Electric Imp
I've been using the Electric Imp in a number of projects. Now I'm using it in a circuit that also includes an ATtiny85. I thought it would be nice if the Electric Imp could also handle the programming of the ATtiny85.The simple solution would have been to program up something like the logic found in the Adafruit standalone AVR chip programmer and hardcode in the details for the ATtiny85.
But as the big feature of the Electric Imp is its web connectivity I thought it would be nice if the programmer logic could avoid any hand coding and derive the relevant Atmel chip details directly from the information available via the web interface to the contents of the Avrdude SVN repository.
Unfortunately the avrdude.conf file format is fairly ad hoc and not easily consumable by anything other than the parser built into the Avrdude application.
So I decided to parse avrdude.conf and make the content available in a format consumable by pretty much anything - JSON.
And make it available in bite size chunks, e.g. one could request the entire contents of avrdude.conf as JSON (around 545KB) or one could request just some specific subset, e.g. a list of all programmers or all parts, or the details for a specific part (around 4.2KB for something like the ATmega328).
Parts should be queryable by the arbitrary names assigned to them by Avrdude, e.g. "m328" for the ATmega328, or by the 3 byte signature that can be retrieved from any Atmel MCU and is unique to each MCU type.
This could have been done in the cloud based agent logic of an Electric Imp.
However this would have been quite complex in Squirrel (a Lua inspired language with C like syntax), the language used for Imp and agent logic.
So I decided to separate this logic off into an independent REST based service, i.e. make it available in a fashion that's easy for applications to interact with but which human beings can also interact with via a web browser.
As a Java programmer this proved quite simple once I'd worked through:
Jackson is a JSON library for Java. Heroku is a PaaS that makes it extremely easy to make your web service publicly available (the free hobbyist tier provides enough container hours to run a single process essentially non-stop). And Jersey is a RESTful web service framework for Java - their getting started guide very conveniently covers how to get a simple example running on Heroku.
REST allows one to request resources via URLs and to tailor the response according to the format or formats the requester expresses a preference for. E.g. a web browser will implicitly express a preference for some form of HTML while an application can explicitly express a preference for JSON.
So one could request JSON like so:
$ curl -H "Accept: application/json" http://avrdude-conf.herokuapp.com/conf/parts/ids/m328And receive:
{
"id" : "m328",
"desc" : "ATmega328",
"has_debugwire" : true,
"flash_instr" : [ 182, 1, 17 ],
"eeprom_instr" : [ 189, 242, 189, 225, 187, 207, 180, 0, 190, 1, 182, 1, 188, 0, 187, 191, 153, 249, 187, 175 ],
"stk500_devcode" : 134,
"signature" : [ 30, 149, 20 ]
...
And then one could request exactly the same URL using a web browser and get:
The web page includes a brief standard introduction, followed by the JSON shown previously. However this JSON is formatted to be human readable, in particular it may include comments, e.g. for the list of parts the part descriptions are included as comments and for the list of signatures the part names are included as comments, and some of the integer values are displayed as hex, e.g. 0xFF rather than 255. Neither comments nor hex are allowed in valid JSON (so there are no comments and all integers are formatted as decimals if one explicitly requests JSON). At the end of the web page is a summary of the URLs for all available resources.
Try it now - click the following URL to request the details for the ATmega328:
https://avrdude-conf.herokuapp.com/conf/parts/ids/m328
Note: Heroku may be slow to respond initially if no one else has used the service recently as Heroku will put a service to sleep after an hour of no requests if the service has only one dyno (the Heroku unit of scalability) - one dyno is the default and typical for hobbyist services. Once out of sleep the service will respond swiftly to subsequent requests.
The source code for this service is available from GitHub here and can be cloned like so:
$ git clone https://github.com/george-hawkins/avrdude-conf.gitIt can be built like so:
$ mvn package
And then run locally like so:
$ java -cp target/classes:target/dependency/* net.betaengine.avrdude.heroku.Main
Then specify http://localhost:8080/conf as the location in your web browser to access it.
Assuming you've worked through the Getting started with Java on Heroku guide mentioned above and created an Heroku account and installed the Heroku toolbelt etc. then deploying the above service so that it's available from anywhere on the web is trivial.
This was done, after going to the base directory of the cloned git repository, as follows:
$ heroku create avrdude-conf $ git push heroku master $ heroku infoThat's it!
So after doing this the service could be accessed using "avrdude-conf.herokuapp.com" rather than "localhost:8080".
The command heroku create takes a name as an argument (in this case I chose "avrdude-conf"), this name must be unique across all Heroku users (so as you can imagine all the more generic names are already gone). The name will ultimately become a subdomain of herokuapp.com.
You can omit the name argument and Heroku will create a unique name itself, something fairly unusual like "immense-inlet-4196". However if you own your own domain you can map a subdomain (named however you choose) of your domain to this unusual name, as described here, for something more memorable.
Note: the Savannah SVN web interface is extremely slow so there is logic to cache rather than rerequest the underlying avrdude.conf.in file each time it's needed. Heroku automatically provides a small PostgreSQL database with each service created and this is used as a persistent cache.
Notes
- Do I approve of REST? REST works very nicely for this service. It's simple to get going and being able to interact with it via a web browser is very convenient. However on the whole I'm unconvinced by REST. Every few years something comes along in this domain because people believe what exists already has become a monster. I'm sure REST people will argue it's addressing a different problem to say SOAP or CORBA but in a few years I suspect the average REST library will have a similar feature set (because all those features are actually useful in some given scenario). At the moment as people discover they want a particular feature that might have been available in one of the supposedly bloated "old" solutions they have to code up their own ad hoc solutions. Soon standardized solutions to those problems will be included into the REST libraries and things will be little different to earlier "monsters".
- Is Heroku the best PaaS option going? Heroku allows for laziness at the development phase - one can pretty much push something that runs on your local machine straight into a service running in the cloud. That's pretty cool - but for a non-hobby project I think I might prefer to live with the increased restrictions of Google App Engine. I started writing out some reasoning for this but it quickly grew and grew - that's a whole separate topic.
