There are a few ways to get around in The Elder Scrolls Online. The first of which is to use your legs and walk. Jokes aside, that's pretty much the primary method of getting around. There are two alternate methods, one of which is the horse which provides a minor speed increase until you level it up and the other is the wayshrine system.

Horses
We has got into vast detail about horses in TESO before, but in general horses are slow and must be evolved over time. You can get a horse rather early in your adventures, but that doesn't mean it's going to be going rather fast. So I consider horses to be the primary mode of transit, but one of the slowest for the longest time.

Sprinting
By pressing down the SHIFT key by default, you can spring. Which consumes stamina. This is the easiest way to move quickly through the world, but it does put you in a weird disadvantage. First, it consumes stamina so if you have to battle you'll need to combat regen your stamina in order to fight with stamina abilities. Next, stamina only has a finite amount, so once you run out you quit sprinting.

Wayshrines
Wayshrines are littered throughout the world and work in many different ways. The shrines offer a resurrection point if you don't have a soul shard on you. You'll resurrect at the nearest wayshrine after death and from there can continue your journey.

The next use is as a transit station. You can go to a wayshrine and teleport directly to any other wayshrine for FREE. That's right, no cost, nothing. You just have to discover the two wayshrines and you get a free ride. Isn't that crazy?

The final use of a wayshrine is that they can be teleported to while out in the world for a fee. I suggest only doing this is you're rich and lazy, since there is enough of them that it takes all of 30 seconds usually to find one to walk to. You can use the map to interface with remote shrines if you do wish to pay the fee.

Transitus Shrines
In PvP the wayshrines become "Transitus Shrines" which allow you to move from one keep to another. It's actually a network of nodes, if there are two connections to a node available then you can instantly teleport to that node. However, if one of the nodes is under attack, you can't teleport.

So essentially if there are two sister keeps captured and not under siege, then you can teleport to the location. This is part of PvP tactics because controlling which nodes are operational and which ones aren't is critical to being able to control the flow of enemies.