To use a program such as gem, you first need to install it. gem comes bundled with ruby. Also note that windows isn't a first-class citizen when it comes to ruby. Or is it the other way around? Anyway, on windows you kind of get stuck when you want to do something there isn't a button for, while there are lots of tools that let you write a command for whatever you want to do. Add to that the licensing fees of windows along with far more restrictions on what you may do..
I would suggest tossing out windows, but perhaps I wouldn't be realistic. But I do suggest using a virtual machine with any major linux distro (such as linux mint) for your programming needs.
That said, you can still make most things run on windows, even if it's a little uphill at times.
On linux you'd just ask your package manager to install ruby and you'd be done, most likely you'd encounter apt where it goes like this:
sudo apt-get install ruby
On windows you'll need to visit ruby-installer's website, and then if you need any gems that are written in C you'd need a compiler, I think there's something called devkit to help with that, but even then there are gems that are too specific to linux or rely on libraries that you'd need to obtain.. you're essentially just trying mimic a linux environment, easier to start in that environment.
jekyll, which might be the only thing you're trying to run, isn't officially supported on windows, but they seem to say that it does run there, I suspect that's the usual story.
And also, if you're going to learn more about an operating system (which you may need to in order to get these things to work) then linux is the better environment to learn since it protects your freedom and isn't subject to the whims of ms