Try the TextAlignment property instead of HorizontalAlignment. First one is the alignment of text in the TextBlock, second the alignment of the TextBlock in the parent container.
TextAlignment
HorizontalAlignment
TextBlock
TextBlock.TextAlignment should do what you're asking for.