No, removing the FAT32 partition will not help.
Sound like you have no longer any Linux partitions on your hard drivers? In this case you just need to remove the master boot record of the first hard drive to get rid of grub.
An alternative way clearing the master boot record is using a Linux live CD. Boot up the Linux live CD and open a command shell window. Then run the following command:
dd if=/dev/null of=/dev/sda1 bs=446 count=1
This will remove the master boot record of the first SATA hard drive (= /dev/sda1).