I don’t know about that. In speed chess, you can lose a game just buy running out of time. Outside of speed chess the state of the board is largely dependent on a sequence of events made over time; even if movement in the 2 directions is always instantaneous, each move is a tick of the clock. Like the 2D board space, most (unique) pieces can move in multiple directions, but like time, games only move forward. Take your hand off the piece, and its irreversible: games move only one direction in time.

I’d say time is definitely a component of the game.