Evaluate Trigonometric Expression Given Cotangent Value in Second Quadrant

To find the value of the given expression, let $E = \left( \sin \frac{11x}{2} \right) (\sin 6x - \cos 6x) + \left( \cos \frac{11x}{2} \right) (\sin 6x + \cos 6x)$

Expanding the product, we get: $E = \sin \frac{11x}{2} \sin 6x - \sin \frac{11x}{2} \cos 6x + \cos \frac{11x}{2} \sin 6x + \cos \frac{11x}{2} \cos 6x$

Rearranging the terms: $E = \left( \cos 6x \cos \frac{11x}{2} + \sin 6x \sin \frac{11x}{2} \right) + \left( \sin 6x \cos \frac{11x}{2} - \cos 6x \sin \frac{11x}{2} \right)$

Using the angle difference identities $\cos(A-B) = \cos A \cos B + \sin A \sin B$ and $\sin(A-B) = \sin A \cos B - \cos A \sin B$ with $A = 6x$ and $B = \frac{11x}{2}$: $A - B = 6x - \frac{11x}{2} = \frac{x}{2}$

Thus, the expression simplifies to: $E = \cos \frac{x}{2} + \sin \frac{x}{2}$

We are given that $\frac{\pi}{2} < x < \pi$ and $\cot x = -\frac{5}{\sqrt{11}}$. Since $x$ lies in the second quadrant, $\sin x > 0$ and $\cos x < 0$.

Using the identity $1 + \cot^2 x = \csc^2 x$: $\csc^2 x = 1 + \left(-\frac{5}{\sqrt{11}}\right)^2 = 1 + \frac{25}{11} = \frac{36}{11}$ $\sin x = \frac{\sqrt{11}}{6}$

Since $\frac{\pi}{2} < x < \pi$, we have $\frac{\pi}{4} < \frac{x}{2} < \frac{\pi}{2}$, which means $\cos \frac{x}{2} > 0$ and $\sin \frac{x}{2} > 0$. Thus, $E > 0$.

Now, let us compute $E^2$: $E^2 = \left(\cos \frac{x}{2} + \sin \frac{x}{2}\right)^2 = \cos^2 \frac{x}{2} + \sin^2 \frac{x}{2} + 2\sin\frac{x}{2}\cos\frac{x}{2} = 1 + \sin x$

Substitute $\sin x = \frac{\sqrt{11}}{6}$: $E^2 = 1 + \frac{\sqrt{11}}{6} = \frac{6 + \sqrt{11}}{6} = \frac{12 + 2\sqrt{11}}{12} = \frac{(\sqrt{11} + 1)^2}{12}$

Taking the positive square root: $E = \frac{\sqrt{11} + 1}{\sqrt{12}} = \frac{\sqrt{11} + 1}{2\sqrt{3}}$

Therefore, the correct option is (B).