What if it was symmetrical? If you had two groups of sapient beings whose perceptions of time were perpendicular to each other? Each group would be capable acts which look like time travel from the POV of the other group.