the arduino IDE needs a small piece of software on the microcontroller, the bootloader, to program it without an external programmer. this howto shows how to burn the arduino bootloader on the atmega8 using the stk500.
BURN ON ATMEGA8 USING STK500 AND AVR STUDIO 4

1. SETUP THE STK500

first, connect the RS232 connector to your computer. if you don't have a serial port, simply use a usb serial adapter.

make sure the jumper are all set in default position (as shown in the stk500 user guide).

then, connect the ISP6PIN to the SPROG2 (green), and insert the atmega8 chip in the green socket.

if you want, connect the PORTB to the LEDS. this allows you to see if the bootloader is correctly installed by making sure the LED5 blinks when the board is reset. the blinking means the bootloader is expecting a program from the arduino IDE.


2. BURN WITH AVR STUDIO 4

first, create a new project for the ATMega8. then, select Tools from the top menu, followed by Program AVR, and Connect. In the dialog, simply select STK500 or AVRISP, Auto, and connect.

at this point, you should see the dialog for the stk500 with the Program, Fuses, LockBits, Advanced, Board and Auto tabs.

2.1 BOARD

start by the Board tab. set the VTarget and ARef required by the ATMega8, 5.2v and 5.0v respectively. the arduino runs at 16MHz so set the STK500 Oscillator to 3.69MHz, which seems to be the maximum. then, it says the ISP Frequency must be 1/4 of the clock speed, so set it to 460.8KHz to be safe. If you get an error later on saying something is wrong with the ISP frequency, then make this value smaller till the error goes away. make sure you write the values to the board each time you modify.

2.2 LOCK BITS

then, you must unlock the ATMega8 so it will allow us to write the bootloader by setting the lock bits value to 0xFF. go to the LockBits tab and activate the following three lines:
  • Mode 1: No memory lock features enabled
  • Application Protection Mode 1: No lock on SPM and LPM in application section
  • Boot Loader Protection Mode 1: No lock on SPM and LPM in boot loader section
Click program and you should see that the lockbits were set to 0xFF in the log.

2.3 FUSE BITS

to config the ATMega8, the high fuse bits must be set to 0xCA and the low fuse bit to 0xDF in the Fuses tab. avr studio dooes not let you simply enter the hex values, so activate the following lines:
  • Boot flash section size=512
  • Boot Reset vector Enabled
  • CKOPT fuse
  • Brown-out detection level at VCC=2.7V
  • Ext. Crystal/Resonator High Freq; Start-up time 16K CK + 0ms
Click program, and you should see the written values 0xCA and 0xDF in the log.

2.2 PRORAM

now you can burn the bootloader. in the Program tab, make sure the selected device is the ATMega8 and the programming mode is ISP. you can leave the erase and verify options activated. then, in the flash section, go find the precompiled bootloader (arduino_install_directory/bootloader/ATmegaBOOT.hex). click program, and verify in the log that the bootloader was written correctly.

2.3 LOCK BITS

finally, you need to lock the ATmega8 so that nothing will overwrite the bootloader. go to the LockBits tab and activate the following three lines:
  • Mode 1: No memory lock features enabled
  • Application Protection Mode 1: No lock on SPM and LPM in application section
  • Boot Loader Protection Mode 3: LPM and DPM prohibited in boot loader section
Click program and you should see that the lockbits were set to 0xCF in the log.

2.3 IDE

place the ATMega8 on your arduino board, and you're ready to program you application using the arduino IDE.
REFERENCES
Arduino - Bootloader
Keyspan USB Serial Adapter