AWS Signature 4 SHA256 Key Hashing

Looking at the code in the link Rick provided, it looks to be more simple than the AWS V4 signature.  Here's the steps I see for the Buckaroo use case:

1. Convert the secretKey from string to byte array
2. Convert the dataToSign from string to byte array
3. Use secretKeyBytes to "sign" dataToSignBytes using hmacsha256 (output needs to be byte array).
4. Convert signatureBytes to base64 encoded string (not a hex encoded string like awsV4)

As for the AWS V4 Signature, here is the (verified working) class I came up with to produce the signature hex string.  This string then gets added to the authorization header in the actual request:

package com.commute.aws.signature;

import java.util.Map;

import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;

import org.apache.commons.lang.StringUtils;
import org.apache.log4j.Logger;

import com.appiancorp.exceptions.InsufficientPrivilegesException;
import com.appiancorp.services.ServiceContext;
import com.appiancorp.suiteapi.common.exceptions.AppianException;
import com.appiancorp.suiteapi.common.exceptions.ErrorCode;
import com.appiancorp.suiteapi.expression.annotations.Category;
import com.appiancorp.suiteapi.expression.annotations.Function;
import com.appiancorp.suiteapi.expression.annotations.Parameter;
import com.appiancorp.suiteapi.security.external.SecureCredentialsStore;

@Category("awsSignatureFunctionsCategory")
public class AwsSignatureGenerator {

	private static final Logger LOG = Logger.getLogger(AwsSignatureGenerator.class);

	@Function
	public static String getAWSV4Signature(ServiceContext sc, SecureCredentialsStore scs,
			@Parameter String scsExternalSystemKey, @Parameter String scsFieldKey, @Parameter String dateStamp,
			@Parameter String regionName, @Parameter String serviceName, @Parameter String stringToSign)
			throws Exception {

		String key = getCryptoKey(scs, scsExternalSystemKey, scsFieldKey);

		/* 1. Get signature key */
		byte[] signatureKey = getSignatureKey(key, dateStamp, regionName, serviceName);
		/* 2. Calculate signature */
		byte[] signature = hmacSHA256(stringToSign, signatureKey);
		/* 3. Encode signature to hex string and return value */
		String stringHexSignature = bytesToHexString(signature);
		return stringHexSignature;
	}

	private static byte[] hmacSHA256(String data, byte[] key) throws Exception {
		String algorithm = "HmacSHA256";
		Mac mac = Mac.getInstance(algorithm);
		mac.init(new SecretKeySpec(key, algorithm));
		return mac.doFinal(data.getBytes("UTF-8"));
	}
	
	private static byte[] getSignatureKey(String key, String dateStamp, String regionName, String serviceName)
			throws Exception {
		byte[] kSecret = ("AWS4" + key).getBytes("UTF-8");
		byte[] kDate = hmacSHA256(dateStamp, kSecret);
		byte[] kRegion = hmacSHA256(regionName, kDate);
		byte[] kService = hmacSHA256(serviceName, kRegion);
		byte[] kSigning = hmacSHA256("aws4_request", kService);

		return kSigning;
	}
	
	private static String bytesToHexString(byte[] bytes) {
		StringBuilder sb = new StringBuilder();
		for (byte b : bytes) {
			sb.append(String.format("%02x", b));
		}
		return sb.toString();
	}

	private static String getCryptoKey(SecureCredentialsStore scs, String scsExternalSystemKey, String scsFieldKey)
			throws AppianException {
		if (StringUtils.isNotEmpty(scsExternalSystemKey)) {
			try {
				// Get Secure Credential Store
				Map<String, String> credentials = scs.getSystemSecuredValues(scsExternalSystemKey);
				if (!credentials.containsKey(scsFieldKey)) {
					LOG.error("Field " + scsFieldKey + " does not exist in Secure Credential Store " + scsExternalSystemKey);
					throw new AppianException(ErrorCode.EXTERNAL_SYSTEM_CONFIGURATION_INVALID_ATTR_NAME, scsFieldKey);
				}
				// Return key
				return credentials.get(scsFieldKey);
			} catch (InsufficientPrivilegesException e) {
				throw new AppianException(ErrorCode.EXTERNAL_SYSTEM_NOT_FOUND_INSUFFICIENT_PRIVILEGES, e);
			}
		} else {
			throw new AppianException(ErrorCode.EXTERNAL_SYSTEM_NOT_FOUND_INSUFFICIENT_PRIVILEGES,
					scsExternalSystemKey);
		}
	}
}

For creating the "string to sign"  here is the expression rule we are using:

/* 
  Steps for creating an AWS V4 Signature:
    1. Create a Canonical Request: https://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html
       Canonical Request Format = 
        [HTTPMethod]\n
        [CanonicalURI]\n
        [CanonicalQueryString]\n
        [CanonicalHeaders]\n
        [SignedHeaders]\n
        [HashedPayload]
        
    2. Create a String to Sign: https://docs.aws.amazon.com/general/latest/gr/sigv4-create-string-to-sign.html
       StringToSign =
        Algorithm]\n
        [RequestDateTime]\n
        [CredentialScope]\n
        [HashedCanonicalRequest]
        
    3. Calculate the Signature: https://docs.aws.amazon.com/general/latest/gr/sigv4-calculate-signature.html
    
    4. Create Authorization Header value: https://docs.aws.amazon.com/general/latest/gr/sigv4-add-signature-to-request.html
       [algorithm] Credential=[accessKeyId]/[credentialScope], SignedHeaders=[signedHeaders], Signature=[signature]
*/
with(
  local!dateStamp: rule!TRSY_formatXAmzDateTime(dateTime: ri!dateTime, isDateOnly: true),
  local!dateTimeStamp: rule!TRSY_formatXAmzDateTime(dateTime: ri!dateTime),
  /* CredentialScope = [dateStamp]/[aws-region]/[aws-service]/aws4_request */
  local!credentialScope: joinarray(
    {
      local!dateStamp,
      cons!TRSY_SQS_AWS_REGION,
      cons!TRSY_SQS_SERVICE_NAME,
      cons!TRSY_SQS_AWS_SIGNATURE_VERSION
    },
    "/"
  ),
  /* Step 1: Canonical request */
  local!canonicalRequest: with(
    local!hashedPayload: sha256hash(ri!body),
    local!headers: joinarray({
      "content-length:"&lenb(ri!body),
      "content-type:text/plain; charset=UTF-8",
      "host:"&cons!TRSY_SQS_HOST,
      "x-amz-content-sha256:" & local!hashedPayload,
      "x-amz-date:" & local!dateTimeStamp
    }, char(10)),
    joinarray(
      {
        "POST",
        cons!TRSY_SQS_URI,
        "Action="&ri!actionParameter&"&MessageGroupId="&ri!messageGroupIdParameter,
        local!headers&char(10),
        cons!TRSY_SQS_AWS_SIGNED_HEADERS,
        local!hashedPayload
      },
      char(10)
    )
  ),
  /* Step 2: String to sign */
  local!stringToSign: joinarray(
    {
      cons!TRSY_SQS_SIGNING_ALGORITHM,
      local!dateTimeStamp,
      local!credentialScope,
      sha256hash(local!canonicalRequest)
    },
    char(10)
  ),
  /* Step 3: Signature */
  local!signature: getawsv4signature(
    scsExternalSystemKey: cons!TRSY_SQS_SCSFIELD_EXTERNALFIELD, 
    scsFieldKey: cons!TRSY_SQS_SCSFIELD_FIELDNAME_SECRETACCESSKEY, 
    dateStamp: local!dateStamp, 
    regionName: cons!TRSY_SQS_AWS_REGION, 
    serviceName: cons!TRSY_SQS_SERVICE_NAME, 
    stringToSign:local!stringToSign
  ),
  /* Step 4: Authorization header value */
  concat(
    cons!TRSY_SQS_SIGNING_ALGORITHM,
    " ",
    joinarray(
      {
        "Credential="&cons!TRSY_SQS_SECRET_ACCESS_KEY_ID&"/"&local!credentialScope,
        "SignedHeaders="&cons!TRSY_SQS_AWS_SIGNED_HEADERS,
        "Signature="&local!signature
      },
      ", "
    )
  )
)

The TRSY_formatXAmzDateTime rule is just a helper to format the date/time into the format AWS requires:

//TRSY_formatXAmzDateTime
if( 
  or(
    rule!APN_isBlank(ri!isDateOnly),
    not(ri!isDateOnly)
  ),
  text(gmt(ri!dateTime), "yyyymmddThhmmss")&"Z",
  text(gmt(ri!dateTime), "yyyymmdd")
)

I think I should be able to add that capability to the cryptography hash plugin. I don't want it to get too unwieldy, but at the same time, it seems nice to have these things in one centralized location, and by adding it to the app market plugin, it promotes easy reuse by other members of the community. 

To that end, can you confirm that the four steps that Josh outlined sound like the right steps for you? If so, I'll go ahead and add a function that does what he described. 

Also, I've updated the cryptography plugin with the last step of the AWS signature process, so if anyone is looking for that, keep an eye out for version 2.2.0 or above.  (jeffreyl946 , that will be the version that you probably want. once it's ready, you can deploy it to your cloud sites via the admin console.) I wound up making the method ever so slightly different than the example Josh gave, so you may need to modify your own SAIL code slightly if you're using Josh's as a reference.

I think the steps that Josh describes for me are correct. If we can have an addition on the current plugin that would be great. 

Sounds great! Version 2.3.0 should be available soon on the App Market and for deployment to cloud sites. That will have an "hmacsha256bytehash" method that will hopefully meet your needs (as well as a getAwsV4Signature, for those who needed an additional function to complete the aws workflow).

It works, thanks for you help! For the response back from Buckaroo I also need to decrypt (un-hash) the message. And the new challenge; they want to send a a-synchronized REST call back with the payment update without any basic authentication only with the hashed body, I'm trying to convince Buckaroo to use basic authentication.

Glad to hear the outbound call is working!! What sort of function would you need in order to decrypt the response? 

For the second part, I think you're correct that you'll still need authentication for any incoming calls to Appian that aren't a direct response to a synchronous call. 

As an update to this post, we're excited to announce that we've added support for AWS Signature Version 4 authentication to HTTP connected systems in Appian. You should no longer require the plugins in order to connect Appian to an Amazon service which uses the AWS Signature Version 4 authentication.

Let us know if you have any questions or concerns!

Hi Tejas Sakhardande,

I am experiencing some difficulties connecting to AWS SQS using new AWS Signature Version 4 authentication.

It looks like Service part of the signature (extracted from ARN) is not correct.

My ARN: arn:aws:sqs:eu-central-1:XXXXXXXXXXXX:sqsname.fifo

The error message: <Error><Type>Sender</Type><Code>SignatureDoesNotMatch</Code><Message>Credential should be scoped to correct service: 'sqs'. </Message><Detail/></Error>

Hi,
Can you let me know what is the value of the 'Service' field in your Connected System?

Hi,

value is full ARN string:  "arn:aws:sqs:eu-central-1:XXXXXXXXXXXX:sqsname.fifo"

I have replaced here my AWS account id with X's.