Rytm tempo varies when USB-synced to Ableton

When I set a tempo in Ableton and press the tempo-button on my Rytm I can see that the tempo fluctuates quite a lot. Say I set the Ableton tempo to 128 my Rytm displays in between 126.5 to 129.5. Also, if I record a regular kick on every fourth step and later look at the recording in ableton I see that it is randomly off. It is not a constant sync delay. Some beats are on the line and others are off.

Anyone else experience this? How can it get resolved?

I posted about something similar recently. Ableton Live + MIDI clock + sync devices. It has some info you might find useful:

Peace,
Andy.

Thanks!