I went through a varicocele surgery about year and a half ago, it was a severe level 3 varicocele, even though it didn't hurted i decided to have it fixed because of several reasons , like fertility and aesthetics (my scrotum was hanging soo low, like and old man's) , the surgery is classified as 'simple' 'not risky' , however I recommend trusting your balls to a good Dr.
I was able to walk properly in 2 days ,and got back to the gym in 2 weeks (as the Dr said), your scrotum is going to get swollen as a coconut, and i didn't felt comfortable jackin off in those 2 weeks ,didn't felt like jelqing for a whole month cause I feared it would fuck up the surgery
However, if you have a level 1 varicocele its not necessary to fix it, level 2 maybe, level 3 preferably