That happens when there are other input devices attached to the computer, typically steering wheel and pedals. Unity is interpreting some arbitrary axis as the "Horizontal" input. Most axes return zero, which represents "full left" in the Horizontal axis.
The solution is reconfiguring the second "Horizontal" entry in Project Settings > Input Manager. Instead of "Get Motion From All Joysticks", choose any entry for a non-existent joystick, i.e. "Joystick 8". You may do it also with the second "Vertical" entry, to prevent any other forced inputs coming from joystick devices.
The emergency stop button typically just disables the force feedback, but it doesn't switch off the device. To be actually disabled, it must be unplugged either from the USB port or from the power source.