Joel Snow has instructions at http://www-hep.nhn.ou.edu/videoconf/polycom-vrvs.txt, once I've tried this, I'll put some additional info here.

Quote from that page:

To bridge an AdHoc meeting with a VRVS meeting:
----------------------------------------------
Click the "Client Setup" tab in the VRVS window, then 
click the "H.323" radio button, then check the "MCU" button on the right,
then make sure the "Gatekeeper" button is checked there and the Gatekeeper 
you are using is 198.128.3.86 (see http://www.ecs.es.net/ for details 
on how to register and use this service) and then enter 8836727 (88DOSAR) -- 
or whatever 88... number you're trying to connect to -- in the 
"Host IP/Alias" field. Note: 82 - 85 numbers don't seem to work with VRVS.

Then click the "Main" tab again, and then "Connect H.323".
Then you should see a new video and audio participant, which will always
show whoever is speaking in the AdHoc meeting.

And of course other AdHoc participants just have to dial 8836727 on their 
polycom.

And if someone wants to call in on the phone, it's 510-883-7860,
and again dial 8836727, and then #.

The attached image shows these settings. Note the icon for Tom Rockwell in that image, it shows that the connection is "MCU H.323" instead of a regular audio and video client.

-- TomRockwell - 11 Sep 2006