You didn't mention your age. On top of Mahhong's excellent reply, I would add the age factor. If you're losing hair at a younger age, and it's progressing quite fast, then this could mean you're less than an ideal candidate.
Hairloss is unpredictable, with some men continuing to lose hair at a steady rate until they are a NW6 or 7, while others might lose a bit, be given a reprieve for a number of years, and then have the process pick up again. Of course the medications, propecia especially, can really help slow down this loss or put a halt to it (at least for a while.)
Sometimes if you suspect that you might one day progress to a higher Norwood number, it's best to not be too greedy. It's possible to use grafts sparingly to create an overall more pleasing look without dropping the hairline down too low.
Do you have any photos to share?