In today's digital age, financial technology (Fintech) applications require robust, scalable, and secure infrastructures to handle complex transactions and ensure smooth payment processing. AWS Step Functions and AWS Serverless Application Model (SAM) provide powerful tools to orchestrate and manage these workflows seamlessly. This hands-on lab will guide you through building a payment processing workflow for a Fintech app using AWS Step Functions and AWS SAM. By the end of this tutorial, you'll have a comprehensive understanding of how to implement a serverless payment processing system that integrates various AWS services to create an efficient and reliable solution.

It is important to note that this is a proof-of-concept solution. A more established method of building payment processing systems on AWS can be found here.

Serverless Payment Processing Orchestration

Overview of The Architecture

In this lab, we will create a payment processing workflow that is both scalable and maintainable, leveraging the power of AWS Step Functions and AWS SAM. Our workflow will involve multiple AWS services, including API Gateway, Lambda, DynamoDB, and SNS, to ensure a seamless transaction processing system. You might want to read this article on step functions first, before proceeding with this lab.

How the Workflow Works

1. API Gateway:

• Acts as the entry point for all client requests.
• Triggers the payment validation process.

2. AWS Step Functions:

• Orchestrates the entire payment processing workflow.
• Manages the sequence of Lambda function invocations.
• Provides error handling and retry mechanisms.

3. AWS Lambda Functions:

• ValidatePayment: Validates the payment details received from the client.
• ProcessPayment: Processes the payment using the validated details.
• UpdateOrderStatus: Updates the order status in DynamoDB.
• SendNotification: Sends a notification about the payment status using SNS.

To learn about AWS lambda's advanced features, read this comprehensive guide on AWS Lambda.

4. Amazon DynamoDB:

• Stores order information and payment status.

5. Amazon SNS:

• Sends notifications to users regarding the payment status (success or failure).

How AWS Step Functions Orchestrates the Workflow

AWS Step Functions play a crucial role in orchestrating the entire workflow. Here's a detailed breakdown of how Step Functions manage the process:

1. Initial Trigger:

• The process begins when a request is received via the API Gateway.
• API Gateway invokes the Step Functions state machine.

2. ValidatePayment State:

• The first state in the state machine invokes the `ValidatePayment` Lambda function.
• This function checks the payment details for validity.
• If validation fails, the workflow can handle the error gracefully, retry, or terminate as defined in the state machine.

3. ProcessPayment State:

• Upon successful validation, the state machine transitions to the `ProcessPayment` state.
• The `ProcessPayment` Lambda function handles the actual payment processing.
• Any errors during this step are managed by the state machine’s retry and catch configurations.

4. UpdateOrderStatus State:

• After processing the payment, the workflow moves to the `UpdateOrderStatus` state.
• The `UpdateOrderStatus` Lambda function updates the DynamoDB table with the order status.

5. SendNotification State:

• Finally, the state machine transitions to the `SendNotification` state.
• The `SendNotification` Lambda function publishes a message to the SNS topic to notify users of the payment status.

6. Error Handling and Retries:

• Throughout the workflow, Step Functions manage error handling and retries.
• Each state can have specific retry and catch configurations to handle transient errors and ensure robust execution.

Implementing the Workflow

We will implement this workflow in several steps:

1. Setting Up AWS SAM Project:

• Create an AWS SAM project to define our serverless application.

2. Creating API Gateway and Lambda Functions:

• Define API Gateway and Lambda functions using AWS SAM templates.
• Implement Lambda functions in Python for validating payments, processing payments, updating order status, and sending notifications.

3. Configuring DynamoDB and SNS:

• Set up a DynamoDB table to store order details and payment status.
• Configure an SNS topic to handle notifications.

4. Defining the Step Functions State Machine:

• Create a state machine definition in the AWS SAM template.
• Specify the sequence of Lambda function invocations and error handling mechanisms.

5. Deploying the Application:

• Deploy the serverless application using AWS SAM.
• Ensure all resources are correctly set up and connected.

6. Testing the Workflow:

• Host a static website on S3 to interact with the API Gateway.
• Use the web interface to test the payment processing workflow.

Step 1: Create The SAM Directory Structure

You have to create the directory structure wherein your sam template and lambda code will exist.

mkdir -p payment-processing/src

Our lambda source code will reside in the src folder.

Step 2: Create The Lambda Functions' Code

The validate_payment lambda function is responsible for validating the credit card details.

src/validate_payment.py:

import boto3
import json
import os


def lambda_handler(event, context):
    print("received event:", json.dumps(event))
    # Validate the payment details from the event
    payment_details = event['payment_details']
    print("Payment Details:", payment_details)
    if not validate_payment(payment_details):
        raise Exception("Invalid Payment Details")

    return {
        'statusCode': 200,
        'body': json.dumps({
            'message': 'Payment validation successful. Workflow initiated.'
        })
    }

def validate_payment(details):
    # Add your payment validation logic here
    if not details.get('card_number') or not details.get('expiry_date') or not details.get('cvv'):
        return False
    return True

The process_payment lambda function is responsible for processing the payment upon validation. Notice that in the source code below, we inroduced logic for random validation error. This will cause our state machine to trigger the handle_payment_error function anytime this transient error occurs. This will occur o.3 (30% or 3/10) of the time.

src/process_payment.py:

import json
import random

def lambda_handler(event, context):
    try:
        # Simulate payment processing logic
        payment_info = event.get('payment_info')
        print(f"Processing payment: {payment_info}")
        
        # Simulate a transient error with 30% probability
        if random.random() < 0.3:
            raise Exception("TransientError")
        
        return {
            'statusCode': 200,
            'body': json.dumps("Payment processed successfully")
        }
    except Exception as e:
        raise e

The update_payment_status function is responsible for persisting the status of the payment in a DynamoDB table. This is neccesary becase you want to persist the status of each transaction you process - both for compliance and quality assurance purposes.

src/update_payment_status.py:

import json
import boto3
import uuid
from botocore.exceptions import ClientError

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('Orders')

def lambda_handler(event, context):
    try:
        # Generate a random payment ID using uuid
        payment_id = str(uuid.uuid4())
        
        # Retrieve status from the event
        status = event.get('status', 'Processed')
        
        # Update the order status in DynamoDB with the generated payment_id
        response = table.update_item(
            Key={'PaymentId': payment_id},  # Assuming 'PaymentId' is your primary key
            UpdateExpression='SET PaymentStatus = :val',
            ExpressionAttributeValues={':val': status}
        )
        
        return {
            'statusCode': 200,
            'body': json.dumps(f"Payment ID {payment_id} status updated to {status} successfully")
        }
    except ClientError as e:
        print(e.response['Error']['Message'])
        raise e
    except Exception as e:
        print(str(e))
        raise e

The send_notification.py will update the customer with the status of their payment. We are using SNS here, but ideally you should use SES or connect to an external provider in a production scenario.

N.B: YOU MUST UPDATE YOUR ARN IN THE CODE BELOW. All you need here is to replace the AccountID in the code with your actual AWS AccountID.

src/send_notification.py:

import json
import boto3
from botocore.exceptions import ClientError

sns_client = boto3.client('sns')
topic_arn = 'arn:aws:sns:us-east-1:123456789012:OrderNotifications' #UPDATE YOUR ACCOUNTID

def lambda_handler(event, context):
    try:
        # Retrieve order ID and status from the event
        order_id = event.get('order_id')
        status = event.get('status', 'Processed')
        
        # Create the message to send
        message = f"Order {order_id} status has been updated to {status}."
        
        # Publish the message to the SNS topic
        response = sns_client.publish(
            TopicArn=topic_arn,
            Message=message,
            Subject="Order Status Update"
        )
        
        return {
            'statusCode': 200,
            'body': json.dumps(f"Notification sent for order {order_id}")
        }
    except ClientError as e:
        print(e.response['Error']['Message'])
        raise e
    except Exception as e:
        print(str(e))
        raise e

The handle_payment_error function will handle any possible errors arising from the payment processing. It is triggered by our State machine once an error is noticed in the ProcessPayment state.

src/handle_payment_error.py:

import json

def lambda_handler(event, context):
    try:
        # Log the error and perform error handling
        print("Handling payment error for event:", event)
        
        # Simulate error handling logic
        return {
            'statusCode': 200,
            'body': json.dumps("Payment error handled successfully")
        }
    except Exception as e:
        print(str(e))
        raise e

Step 3: Define The State Machine Workflow

We will be defining our state machine using the Amazon States Language (ASL). It is a JSON-based, structured language used to define a step functions state machine.  Take note of the code below. We will be integrating it directly into our SAM template.

{
          "Comment": "Payment processing workflow triggered via API Gateway",
          "StartAt": "ValidatePayment",
          "States": {
            "ValidatePayment": {
              "Type": "Task",
              "Resource": "${ValidatePaymentFunction.Arn}",
              "Next": "ProcessPayment"
            },
            "ProcessPayment": {
              "Type": "Task",
              "Resource": "${ProcessPaymentFunction.Arn}",
              "Retry": [
                {
                  "ErrorEquals": ["TransientError"],
                  "IntervalSeconds": 5,
                  "MaxAttempts": 3,
                  "BackoffRate": 2
                }
              ],
              "Catch": [
                {
                  "ErrorEquals": ["States.ALL"],
                  "Next": "HandlePaymentError"
                }
              ],
              "Next": "UpdatePaymentStatus"
            },
            "UpdatePaymentStatus": {
              "Type": "Task",
              "Resource": "${UpdatePaymentStatusFunction.Arn}",
              "Next": "SendNotification"
            },
            "SendNotification": {
              "Type": "Task",
              "Resource": "${SendNotificationFunction.Arn}",
              "End": true
            },
            "HandlePaymentError": {
              "Type": "Task",
              "Resource": "${HandlePaymentErrorFunction.Arn}",
              "Next": "UpdatePaymentStatus"
            }
          }
        }

Step 4: Create The SAM Template

The SAM template is an open-source framework that you can use to define and manage your serverless application infrastructure code.

template.yaml:

AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'

Resources:
  ValidatePaymentFunction:
    Type: 'AWS::Serverless::Function'
    Properties:
      Handler: validate_payment.lambda_handler
      Runtime: python3.12
      CodeUri: src/
      Policies:
        - AWSLambdaBasicExecutionRole

  ProcessPaymentFunction:
    Type: 'AWS::Serverless::Function'
    Properties:
      Handler: process_payment.lambda_handler
      Runtime: python3.12
      CodeUri: src/
      Policies: AWSLambdaBasicExecutionRole

  UpdatePaymentStatusFunction:
    Type: 'AWS::Serverless::Function'
    Properties:
      Handler: update_payment_status.lambda_handler
      Runtime: python3.12
      CodeUri: src/
      Policies: 
        - AWSLambdaBasicExecutionRole
        - Version: '2012-10-17'
          Statement:
            - Effect: Allow
              Action:
                - 'dynamodb:PutItem'
                - 'dynamodb:UpdateItem'
              Resource: !GetAtt OrdersTable.Arn

  SendNotificationFunction:
    Type: 'AWS::Serverless::Function'
    Properties:
      Handler: send_notification.lambda_handler
      Runtime: python3.12
      CodeUri: src/
      Policies: 
        - AWSLambdaBasicExecutionRole
        - Version: '2012-10-17'
          Statement:
            - Effect: Allow
              Action:
                - 'sns:List*'
                - 'sns:Publish'
              Resource: !Ref OrderNotificationsTopic  
          

  HandlePaymentErrorFunction:
    Type: 'AWS::Serverless::Function'
    Properties:
      Handler: handle_payment_error.lambda_handler
      Runtime: python3.12
      CodeUri: src/
      Policies:
        - AWSLambdaBasicExecutionRole

  OrdersTable:
    Type: 'AWS::DynamoDB::Table'
    Properties:
      TableName: Orders
      AttributeDefinitions:
        - AttributeName: PaymentId
          AttributeType: S
      KeySchema:
        - AttributeName: PaymentId
          KeyType: HASH
      ProvisionedThroughput:
        ReadCapacityUnits: 5
        WriteCapacityUnits: 5

  OrderNotificationsTopic:
    Type: 'AWS::SNS::Topic'
    Properties:
      TopicName: OrderNotifications

  PaymentProcessingStateMachine:
    Type: 'AWS::StepFunctions::StateMachine'
    Properties:
      DefinitionString: !Sub |
        {
          "Comment": "Payment processing workflow triggered via API Gateway",
          "StartAt": "ValidatePayment",
          "States": {
            "ValidatePayment": {
              "Type": "Task",
              "Resource": "${ValidatePaymentFunction.Arn}",
              "Next": "ProcessPayment"
            },
            "ProcessPayment": {
              "Type": "Task",
              "Resource": "${ProcessPaymentFunction.Arn}",
              "Retry": [
                {
                  "ErrorEquals": ["TransientError"],
                  "IntervalSeconds": 5,
                  "MaxAttempts": 3,
                  "BackoffRate": 2
                }
              ],
              "Catch": [
                {
                  "ErrorEquals": ["States.ALL"],
                  "Next": "HandlePaymentError"
                }
              ],
              "Next": "UpdatePaymentStatus"
            },
            "UpdatePaymentStatus": {
              "Type": "Task",
              "Resource": "${UpdatePaymentStatusFunction.Arn}",
              "Next": "SendNotification"
            },
            "SendNotification": {
              "Type": "Task",
              "Resource": "${SendNotificationFunction.Arn}",
              "End": true
            },
            "HandlePaymentError": {
              "Type": "Task",
              "Resource": "${HandlePaymentErrorFunction.Arn}",
              "Next": "UpdatePaymentStatus"
            }
          }
        }
      RoleArn: !GetAtt StepFunctionsExecutionRole.Arn

  StepFunctionsExecutionRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: 'states.amazonaws.com'
            Action: 'sts:AssumeRole'
      Policies:
        - PolicyName: StepFunctionsExecutionPolicy
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - 'lambda:InvokeFunction'
                Resource:
                  - !GetAtt ValidatePaymentFunction.Arn
                  - !GetAtt ProcessPaymentFunction.Arn
                  - !GetAtt UpdatePaymentStatusFunction.Arn
                  - !GetAtt SendNotificationFunction.Arn
                  - !GetAtt HandlePaymentErrorFunction.Arn

  PaymentAPI:
    Type: AWS::Serverless::Api
    Properties:
      StageName: Prod
      DefinitionBody:
        swagger: '2.0'
        info:
          title: 'PaymentAPI'
          version: '1.0'
        paths:
          /startPayment:
            post:
              summary: 'Initiate Payment Processing'
              description: 'Initiate the payment processing workflow in Step Functions.'
              consumes:
                - 'application/json'
              produces:
                - 'application/json'
              parameters:
                - in: 'body'
                  name: 'body'
                  required: true
                  schema:
                    type: 'object'
                    properties:
                      payment_details:
                        type: 'object'
                        properties:
                          card_number:
                            type: 'string'
                            description: 'The card number for payment.'
                          expiry_date:
                            type: 'string'
                            description: 'The expiry date of the card (MM/YY format).'
                          cvv:
                            type: 'string'
                            description: 'The CVV code of the card.'
                          amount:
                            type: 'number'
                            description: 'The amount to be paid.'
              responses:
                '200':
                  description: 'Successful response'
                  schema:
                    type: 'object'
                    properties:
                      statusCode:
                        type: 'integer'
                      body:
                        type: 'string'
              x-amazon-apigateway-integration:
                type: 'AWS'
                httpMethod: 'POST'
                uri: !Sub 'arn:aws:apigateway:${AWS::Region}:states:action/StartExecution'
                credentials: !GetAtt PaymentAPIGWRole.Arn
                requestTemplates:
                  application/json: !Sub |
                    {
                      "stateMachineArn": "${PaymentProcessingStateMachine}",
                      "input": "$util.escapeJavaScript($input.json('$'))"
                    }
                responses:
                  default:
                    statusCode: '200'
                    responseTemplates:
                      application/json: |
                        {
                          "statusCode": 200,
                          "body": $input.json('$.output')
                        }

  PaymentAPIGWRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: 'apigateway.amazonaws.com'
            Action: 'sts:AssumeRole'
      Policies:
        - PolicyName: ApiGatewayInvokeStepFunctionsPolicy
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action: 'states:StartExecution'
                Resource: !Ref PaymentProcessingStateMachine

Outputs:
  ApiUrl:
    Description: "API Gateway endpoint URL for Prod stage"
    Value: !Sub "https://${PaymentAPI}.execute-api.${AWS::Region}.amazonaws.com/Prod/startPayment"

Step 5: Build and Deploy the Application

 1. Build the App:

sam build

 2. Deploy the App:

sam deploy --guided

As you can see, our state machine was successfully created.

Step 6: Test the Application

Now, the hard work has been done. What remains for us is to tets our s3 bucket and make sure that this setup is working as smoothly as intended.

There are several ways of tesing this; using curl, using postman or creating a frontend through which we can query our backend api.

• Using Curl:

curl -X POST https://<your-api-id>.execute-api.<region>.amazonaws.com/Prod/startPayment -d '{
  "payment_details": {
    "card_number": "4111111111111111",
    "expiry_date": "12/24",
    "cvv": "123",
    "amount": 100.0
  }
}' -H "Content-Type: application/json"

• Using Postman:

My 'Orders' table also clearly shows that the payment information is beign persisted in the table.

Here you can also see that the workflow executed successfully:

If I were to enter an invalid card detail such as a detail without "cvv" or "card number", the workflow fails.

Final Thoughts

If you made it this far, CONGRATULATIONS!!! You Rock!!!

By following this tutorial, you have successfully built a comprehensive payment processing workflow for a Fintech app using AWS Step Functions and AWS SAM. This serverless architecture not only ensures scalability and reliability but also simplifies maintenance and reduces operational overhead. You have integrated multiple AWS services to create an efficient system that handles payment validation, processing, status updates, and notifications. This hands-on experience equips you with the skills to implement similar workflows for other complex applications, leveraging the full potential of AWS's serverless capabilities. As you continue to explore and innovate, you'll find even more ways to optimize and enhance your Cloud solutions as the quintessential Practical Cloud Engineer.

Here's a simillar project using step functions to implement a user onboarding orkflow.

Click on this GitHub repo to get all the source code used in this project.

Happy Clouding!!!