Babel seems counterproductive

Babel seems like it would take lightweight modern JS code and transpile it into files that will take longer to load and provides an opportunity for glitches to occur.

Is it being suggested that we not actually use our ES6 code and always transpile it to ES5?

It just seems backwards. Why can’t the code be transpiled in real-time and sent if requested by an older browser?
Or, have the new code and the transpiled code available on the server and when the request is made, send the appropriate file?

I guess I’m confused on the ‘big picture’ benefit of this.

Any insight would be appreciated.