You have the right idea guest, here is how you solve it: \(x+y+\sqrt {x^2+y^2}=30\)
 Subtarct \(x+y\) from both sides: \(\sqrt{x^2+y^2}=30-x-y\)
 Square both sides: \(x^2+y^2=x^2+y^2+2xy-60x-60y+900\)
 Subtract \(x^2+y^2\) from both sides:\(0=2xy-60x-60y+900\)
 Subsitute \(xy\) for 60:\(1020-60x-60y=0\)
 Rewrite as: \(60x+60y=1020\)
 Factor the left side: \(60(x+y)=1020\)
 Divide both sides by 60: \(x+y = 17\)
 From the other equation, divide by \(y\): \(x = {60\over y}\)
 Plug into the other equation: \({60\over y}+y=17\)
 Multiply by \(y\): \(60+y^2=17y\)
 Subtract \(17y\) from both sides: \(y^2-17y+60=0\)
 Use quadratic formula, and you find \(y = 12\)
 Subsitute into the other equation, and \(x = 5\)
 This means that the third side is \(13\), since the triangle must have a perimeter of 30. 
  
 Thus, the side legnths are: \(\color{brown}\boxed{5,12,13}\)
.