Porting Peter Fleury’s STK500V2 bootloader to a custom avr board.

mega128 isp
mega128 accelerometer board with stk200 compatible isp programmer and usb to serial port adapter (rs232 to ttl circuit not shown).

I had problems porting the bootloader to my board twice, once with the atmega128 and now with the atmega168p while it worked already on the atmega128.

And I need a bootloader, for boards hidden from the user in a bag or in a box. I can’t use the SPI there. Plus the classic SPI connector takes a huge surface on the board compared to the chip size. I prefer to program a bootloader once with SPI and then upload the firmware through a serial port.

In this article I will describe the steps to port the bootloader created by Peter Fleury to a custom board. It is a compact bootloader with many stk500v2 functions. It has a lot of configuration options selected by macro definitions.


1/ Get the source code:

Many versions can be found. You can take my copy at github.com/mandraga/SombreroBms in the firmwarebms/stk500v2bootloader/ section (or you can find different bootloaders at github.com/arduino/Arduino).


2/ Change the makefile:

Change the µcontroller name to what you are using:

# MCU name
MCU = atmega128;

Change the clock to your clock:

#         F_CPU =  1000000
#         F_CPU =  1843200
#         F_CPU =  2000000
#         F_CPU =  3686400
#         F_CPU =  4000000
#         F_CPU =  7372800
#         F_CPU =  8000000
#         F_CPU = 11059200
#         F_CPU = 14745600
#         F_CPU = 16000000
#         F_CPU = 18432000
#         F_CPU = 20000000
F_CPU = 7372800

You can notice that most clocks are not simple numbers like 8000000hz. It is because avrdude uses 115200 bauds for the serial communication and in order to transmit data at this baudrate without any error you need a clock like 1843200hz or multiple of it. With this kind of frequency, the serial clock will be at 0,0% difference to the ideal baudrate. But if you use a 16Mhz clock you will have a 2,5% error. And 2,5% is enough to mess with the communication on a serial line (big chances to upload crap). In this case I picked a 7372800hz crystal.

Change the bootloader area start address:

# Bootloader
# Please adjust if using a different AVR
# 0x0e00*2=0x1C00 for ATmega8  512 words Boot Size
# Atmega168PA 0x1E00 512W but avrgcc takes bytes so   3C00
# Atmega128   0xF000 4096W but avrgcc takes bytes so 1E000


On this table from the atmega128 datasheet you can find the flash sections selected with the BOOTSZ1 and BOOTSZ0 fuses. For my atmega128 board I chose the 4096 words (8KBytes) bootloader size. And as you can see here, the bootloader start section begins at $F000. But this address is in words and gcc needs it in bytes. Therefore you must shift it left once to get the address given to gcc. $F000 becomes $1E000.

I made a mistake here and it took me several days to find it (that’s why I post this article). I used $F000 as the bootloader start address in gcc but my bootloader was somehow executing anyway while being in the middle of the 128kBytes of flash. And I could test the stk500v2 protocol, read the fuses… But when it came to use the SPM instruction to erase/program the flash, it failed. I had this message:

avrdude: verifying ...
avrdude: verification error, first mismatch at byte 0x0000
0x0c != 0xff
avrdude: verification error; content mismatch

First I though that the lock fuses were wrong, but they were 0xFF (SPM can read/write anything when executing in the boot section). Then I though that the power went down during programming, but no. And then I though that the avr-libc boot.h headers of the atmega168pa were not compatible. So I wrote the function in assembler. And what lost me is that when downloading the flash memory with the isp programmer, it missed the end of the flash and stopped at the end of the boot code. The downloaded flash content looked like the bootloader was at the end of the flash. Therefore I assumed that my boot address was the right one. This is where I began to lose time double checking everything.

By checking everything I finally changed the gcc code address to 0x1E000. And at this point SPM did erase the memory and write the page content to the flash.

3/ Changing stk500boot.c:

Chose the uart: some atmegas like the atmega128 have 2 uarts. By default it uses UART0 and if you define “#define USE_USART1”, you select the second uart.

The bootloader can use a led to indicate that it is working and an input button to indicate that you want to program the chip. The ports for it are specified here:

* Pin "PROG_PIN" on port "PROG_PORT" has to be pulled low
* (active low) to start the bootloader
* uncomment #define REMOVE_PROG_PIN_PULLUP if using an external pullup
#define PROG_DDR   DDRD
#define PROG_IN    PIND
#define PROG_PIN   PIND2

* Active-low LED on pin "PROGLED_PIN" on port "PROGLED_PORT"
* indicates that bootloader is active
#define PROGLED_PIN  7

I do not use the prog pin and everyone can change this in his own way. Change to your led port and pin here. And depending if it is a source or sink led port, write PROGLED_PORT = (1<<PROGLED_PIN); or PROGLED_PORT &= ~(1<<PROGLED_PIN); at the start of the main.

The flash address:

* Calculate the address where the bootloader starts from FLASHEND and BOOTSIZE
* (adjust BOOTSIZE below and BOOTLOADER_ADDRESS in Makefile if you want to change the size of the bootloader)
//#define BOOTSIZE 512
#define BOOTSIZE 4096
#define APP_END  (FLASHEND -(2*BOOTSIZE) + 1)

Here the value is defined in words, but APP_END is the same value as in the makefile, int Bytes.

4/ In circuit programming

Once you have changed all the parameters and compiled the bootloader, pick your spi programmer and do two things (assuming the clock fuses are already setup):

  •  Set the fuses: BOOTSZ1/0 to the flash size you chose and BOOTRST enabled by setting it to 0 in order to boot to the bootloader memory instead of $0000 on reset.
  • Program the bootloader with the SPI programmer.

At this point your bootloader should be running on the board. If you call avrdude like

avrdude -p atmega128 -P /dev/ttyUSB0 -c stk500v2 -U flash:w:main.hex

it should program the flash.


5/ What if avrdude does not work?

If at this point, avrdude fails you will ask yourself how to check the protocol? And by running “avrdude –help” you see that adding -v -v -v to the command line gives a lot of information. But not enough to debug the protocol.

I had a problem with an stk500v2 command, SPI_MULTI did not pass. A good solution to find a protocol problem is to download avrdude’s source code and in stk500v2.c change this:

#if 0
#define DEBUG(...) fprintf(stderr, __VA_ARGS__)
#define DEBUG(...)

Change #if 0 to #if 1 and recompile (make) reinstall (sudo make install). Now your avrdude will print all the data sent and received to and from the serial port. You will be able compare the data to what is specified in the protocol here: http://www.atmel.com/images/doc2591.pdf

And if avrdude fails somewhere you know on which command it is. In my case I forgot to enable SPI_MULTI by defining #define REMOVE_CMD_SPI_MULTI. You must comment this define otherwise avrdude will fail when sending the SPI_MULTI command. And if you use the default avrdude you will have no way to tell it was this command who failed.

The STK500v2 command and parameter codes can be found in the include file “command.h”.

About binary code size.

On the atmega168pa I need a bootloader size of 512Words, but I have this output with the same code as the atmega128. My binary is 1068Bytes long. Therefore it does not fit the 512W/1024Bytes.

Creating Symbol Table: stk500boot.sym
avr-nm -n stk500boot.elf > stk500boot.sym

Size after:
   text	   data	    bss	    dec	    hex	filename
   1068	      4	      0	   1072	    430	stk500boot.elf

-------- end --------

To go down to 512Workds, I uncommented “#define REMOVE_CMD_SPI_MULTI”. It removes part of the protocol, but it works with avrdude 6.3.

Creating Symbol Table: stk500boot.sym
avr-nm -n stk500boot.elf > stk500boot.sym

Size after:
   text	   data	    bss	    dec	    hex	filename
    962	      4	      0	    966	    3c6	stk500boot.elf

-------- end --------

In this case it was just because I needed all the memory, the BMS serial protocol takes a lot of program space (like 14700B/16KB). On the next version I will use an atmega32C1 and the bootloader size will not be a problem.


Configuring a bootloader for your avr may be difficult but it could be the best choice to program your board. With the condensed info here it should be easier to adapt it.