clk: ti: gate: use custom API for memory access