Oblivious to and oblivious of can both be used the same way. You can generally pick one based on which sounds better in the context of the sentence. Essentially, they both mean that someone is unaware of something.
Oblivious to and oblivious of share a meaning with a number of phrases, like ignorant of, unaware of, unconscious of, and blind to. Similarly, it may be more common to hear and use oblivious of than oblivious to.
Oblivious to and oblivious of can pretty much be used interchangeably. In J. R. R. Tolkien’s The Fellowship of the Ring, Frodo and Bilbo “talked together in soft voices, oblivious of the mirth and music in the hall about them.” Tolkien could have used oblivious to and it would still mean they didn’t notice how loud the room was.
The difference between these two is in the preposition. To is a preposition that tends to identify motion toward something. An example is, “I gave all my love to you.” Of is a preposition that often expresses distance or separation from something, as in “That’s one of my shirts.”
Consider “oblivious to the weather” versus “oblivious of the weather.” Neither phrase is incorrect. They can technically be used interchangeably. But there may be a slight difference between them.
If to means toward, someone who’s “oblivious to the weather” might have gotten a close look at the weather but chosen to disregard it. Maybe they’re just leaving the umbrella at home, and not taking mind to the dark rain clouds.
If of means separate from, then someone who’s “oblivious of the weather” is completely unaware of what the weather’s like outside. Maybe they’ve been inside all day, in a room with no windows, and have no idea it’s been raining.
Since both phrases can be used interchangeably in most cases, it can be left to personal choice which to use in a given sentence. This can be useful if you want to avoid repeating a preposition. For example, rather than use of twice in a sentence like, “He was oblivious of the reactions of his friends,” you could say “He was oblivious to the reactions of his friends.” Neither option is more correct, but you might personally prefer one over the other.