To get rid of the XP visualstyle you would have to comment out the line Application.EnableVisualStyles(); within the main method. which VS automatically adds. but this would affect all the controls in the app.
or else
Set the FlatStyle property of the buttonto FlatStyle.Flat to get rid of the borders.
if that too wont suffice you would have to ownerdraw the button.