I can't believe I am typing a question for a simple thing like this but here we are. I can't for the life of me figure out what the exact name for the settings file is for vim on Windows (.vimrc does not work on Windows). And if I have names
right (read the next para) then I don't know why it is not picking up the settings from it.

I tried creating _vimrc and _gvimrc in the root directory where I copied the file from the example settings file that came installed with vim. I set the following command on top of the example commands:

set nobackup

set nowritebackup

set guifont=Courier_New:h11:cANSI

It is not accepting this file (whether it is named _vimrc or
_gvimrc, I tried both) as the settings file as the font has not changed on the next start-up and still writes backup files.

my _vimrc/_gvimrc is stored at C:/Users/<ME>/_vimrc and is working fine.

It's generally a good idea to keep personal settings separate from installation files.

To get more information about the search paths for your configuration files, type
:he vimrc-intro.

And be careful: the docs say

For MS-DOS and MS-Windows you can use one of these:

$HOME/_vimrc

$VIM/_vimrc

While this is absolutely true, it could be a bit surprising that $VIM does not expand to e.g.
C:/Program Files/vim/<your_version> (this is what $VIMRUNTIME holds) but only to
C:/Program Files/vim/.

This said, C:/Program Files/vim/_vimrc should be read during startup.