I say its the perfect time. I asked my old co-workers and most of them they said they didn't stop growing till their mid twenties. One guy said he stopped when he was 16, he was the only white guy, I don't know if that's a coincidence or what. My uncle said he had a growth spurt on his in his twenties.
Men are different that women, some of us grow all the way to our mid twenties as far as height, bone density, hair and facial structure. I don't see any thing that would suggest our penis stop growing earlier than other part of our body. I believe jelqing will encourage growth in early days as well.
I'm 20 and my face and body makes me look real young, so Ive been told. I feel like I have no real manly physical qualities yet. I just started getting hair under my arms last year, I didn't start getting muscles till I started working out. I had a phyical and Im 6'1 now, just last year I was 6'0. Everyone is different but I'm still changing