Amy, Daniel gave good advice about using a gel to help ease discomfort while getting used to nasal pillows. Ayr gel, KY jelly, and Neosporin ointment are some of the most popular.
Also as Daniel said, using a bigger size of nasal pillows can help - bigger than most DME's would "fit" you with. DME is message board shorthand for the "home health care supplies store" - the people that sell you "Durable Medical Equipment".
The sizes of Breeze nasal pillows are in these colors:
Blue - small
Clear - medium
Light mint green - large
Lavender - extra large
I'm a woman with average size face and smallish nares (the openings to the nostrils) but I find the "Large" nasal pillows very comfortable with the Breeze. "Large" stay more to the outside of my nostrils, sealing very well without having to be crammed up insides the nostrils.
Some find it helpful to rotate between two or more masks that suit you....keeps any one interface from pressing on the same areas, night after night.
If you are leaking treatment air out your mouth - very likely since you described a dry mouth - you really will have to get that problem solved. Gotta get that air going where it can really do the job. Air detouring out the mouth won't keep the throat open. As mesocosm said, you might need to go to a full face mask (covers nose and mouth) if you can't stop the mouth leaks or mouth breathing while wearing a nasal mask or nasal pillows.
A woman on another message board came up with a design for a homemade mouth guard that several, including me, have been able to make for ourselves. Stops our mouth air leak problems and lets us wear whatever mask we choose:
Topic started by frequenseeker Dec 01 2004 subject: mouth leak solution, cheap DIY oral appliance