Duel tournaments are held in another part of the duel arena from normal duels and can only be done in world 54. It works basically the same way as a normal duel except you work your way through several opponents and get all of the entry money at the end of you win.
To get there:
The duel arena is in Al Karid. Most people would use a ring of duelling to teleport themselves straight there. You can also walk from Lumbridge or Varrock if your poor. Here where it is:
How to get to the Runescape Duel Arena
When you get there you will see a red minigame symbol on the map, this is the tournament entrance:
The Duel Tournament Entrance