The simple answer to the question is if you already have the X then it is not worth getting the XS unless you just want to have the latest technology. However if you want a bigger screen then it is a worthwhile upgrade to get the XS Max.
Main differences in the X compared to XS
-Slightly faster Face ID
-30 minutes more battery life
-Available in Gold
If you are coming from any phone other than the X, then it is definitely worth getting the XS, or the XS Max if you want the biggest screen. If you already have the X and don't wan't a bigger screen I would save your money and get an Apple Watch Series 4 instead.