XML Signature not validating when adding an element explicitly to the document (JAVA)?

How can I find the actual XML string being signed by the API, I tried reading the reference input stream after enabling javax.xml.crypto.dsig.cacheReference but this did not work when signing, it only worked when validating?

Fortunately the default implementation, assuming that's being used, offers some debug capabilities. Here's an example of settings to log to the console using the FINEST granularity, although FINE seems to be enough.

handlers= java.util.logging.ConsoleHandlerjava.util.logging.ConsoleHandler.level = FINERjava.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatterorg.jcp.xml.dsig.internal.level = FINERcom.sun.org.apache.xml.internal.security.level = FINER

You can save this as a logging.properties file and provide its path via command line option -Djava.util.logging.config.file, although it can probably also be done programmatically.

Doing so will actually show the canonicalized XML being signed, which answers this:

Is the namespace actually being omitted from the payload digest calculation when adding the element to the document using createElementNS and appendChild?

Indeed, it is. Running the code I saw this in the logging as the canonicalized XML for the signing.

<DataPDU xmlns="urn:swift:saa:xsd:saa.2.0"><Revision>2.0.6</Revision><Saa:LAU></Saa:LAU></DataPDU>

Notice how the Saa prefix is there but the namespace declaration is not. Yet, the info is in there because after transformation the declaration with the prefix shows up in the result. That explains why changes to the namespace URI itself don't result in a different digest but changing the prefix does. No idea why this is happening. It feels like it shouldn't, but the rules for what is included in the canonicalized version are so confusing in the specs that either I'm missing something, or it's a bug in the implementation. A work-around for this is adding this line after creating the LAUElement:

 LAUElement.setAttribute("xmlns:Saa", "urn:swift:saa:xsd:saa.2.0");

I thought maybe the DOMResult used a document builder that wasn't set to be namespace-aware, but I tried this with no difference:

DocumentBuilderFactory domFactory = DocumentBuilderFactory.newInstance();domFactory.setNamespaceAware(true);DocumentBuilder docBuilder = domFactory.newDocumentBuilder();Document doc = docBuilder.newDocument();DOMResult domResult = new DOMResult(doc);marshaller.marshal(myDataPDU, domResult);

It might be a bug of DOMResult itself. The workaround is a bit messed up but it does the job. This does in fact change the digest. I've tried this by providing both the LAUElement and the document root as the input for DOMSignContext, which in both cases results in the same signature (digest and signature value) although in the latter case the signature is added to the root, not the element.

This also teaches us that the entire document is used as input for the signing, even when you supply the element and use the exclusive canonicalization method. It only changes where the signature is placed. The URI resolving finds the ancestors up to the root element and everything below it. This surprised me a bit.

Doing the above, I managed to correctly verify the signature using the below code:

XMLSignatureFactory factory = XMLSignatureFactory.getInstance("DOM");NodeList sigList = doc.getElementsByTagNameNS(XMLSignature.XMLNS, "Signature");if (sigList.getLength() == 0) {    throw new Exception("Cannot find Signature element");}String secretKey = "Abcd1234abcd1234Abcd1234abcd1234";SecretKeySpec secret_key = new SecretKeySpec(secretKey.getBytes(), "HmacSHA256");DOMValidateContext valContext = new DOMValidateContext(secret_key, sigList.item(0));XMLSignature signature = factory.unmarshalXMLSignature(valContext);System.out.println("Core validity: " + signature.validate(valContext));

It does indeed seem that the issue was that when the namespace declaration on was dropped, the digest generated differed from the one generated on validation.

Is there a way to provide through JAXB a prefix for the same root namespace to a single element only?

The short answer is, there's no trivial way to do this. The long answer can be found here: https://stackoverflow.com/a/42301479/630136

EDIT:

An additional note. Be careful with the transformer for outputting the signed document to a file. If it's set to indent the output, the resulting file would actually have a different digest. There's whitespace that can be ignored (like in <empty /> vs <empty/>) but within elements it's normally regarded as text nodes and part of the canonicalized version over which the digest is calculated.