If you completed the side quest named One on One: Vergen, while playing the second chapter of the game, allied with Iorveth, you probably faced a challenger named Silgrat.
He was a nobleman which also appears during your negotiations for Stenniss life (Royal Blood quest).
No matter how you have played One on One: Vergen, you activated a side quest named A Score to Settle.
If you track this side-quest during the second chapter of the game, you will be asked to wait for events to unfold; however, you will have to wait until you play Chapter 3.
“As you may remember, Seltkirk was the hero of the previous war with Kaedwen. Everyone in Vergen had heard of him, but his brother Silgrat was no nearly as famous.
Living in the shadow of his great brother, Silgrat became embittered, hungry for acclaim. He thought defeating a witcher in a fistfight would bring him glory. Geralt picked up the gauntlet.”
When to start
After One on One: Vergen, but you can finish this quest in the third act
The quest continues in Loc Muinne, where you will find Silgrat in the camp located to the north from the main gate.
The city is in ruins, so search the remaining buildings, especially those near the stone ramps.
When you find Silgrat, defeat him again in a fist-fight to complete the quest.