In Octatrack’s studio mode you can use its two pairs outputs as separate. So, you would create a thru machine for you effects send signal, and route back the output via one of the output pairs. The other output pair could be used for “other OT stuff”. See manual page 140 for an example, and page 36 on Studio mode.
Yes, easily, you can also (as an option to studio mode) just cue the wet only effects from the track assigned to be an aux fx, this leaves you 7 other OT tracks to do with as you please (using main output), or 6 etc if you want to extend the FX loop to two tracks using a Neighbour after the Thru (i.e. 4 series FX ‘plugins’)