Integrating Magento with NetSuite via AWS Middleware

Preface

This project involved designing and implementing a secure and scalable AWS middleware solution, enabling seamless integration between the Magento e-commerce platform and NetSuite's ERP system. The middleware streamlined data synchronisation and automated key business processes.

Tech Stack: AWS CDK, AWS Lambda, AWS Step Functions, AWS SQS FIFO, AWS API Gateway, AWS Secrets Manager, Magento, NetSuite, TypeScript

Project Overview

I developed a middleware solution to bridge Magento’s e-commerce functionality with NetSuite’s ERP system, automating workflows such as order processing, inventory management, and financial reconciliation. The solution provided significant efficiency gains, reduced manual intervention, and increased data accuracy.

Technical Challenges & Solutions

Ensuring Sequential Data Integrity

Implemented AWS SQS FIFO queues to maintain accurate order processing and data integrity:

const fifoQueue = new sqs.Queue(this, 'FifoQueue', {
  fifo: true,
  contentBasedDeduplication: true,
});

Managing Throttling Limitations

Managed API load and prevented system overloads using AWS Lambda concurrency limits:

const lambdaFunction = new lambda.Function(this, 'ProcessFunction', {
  runtime: lambda.Runtime.NODEJS_16_X,
  handler: 'index.handler',
  reservedConcurrentExecutions: 10,
});

Custom Authentication Solution

Built a tailored authentication mechanism using AWS Lambda and Secrets Manager to ensure secure API Gateway communications:

export async function handler(event: any, context: object) {
  const secrets = await getSecrets();
  const authorised = event.key === secrets.platformKey;

  return {
    principalId: "apigateway.amazonaws.com",
    policyDocument: {
      Version: "2012-10-17",
      Statement: [
        {
          Action: "execute-api:Invoke",
          Effect: authorised ? "Allow" : "Deny",
          Resource: event.methodArn,
        },
      ],
  };
}

Securing Endpoint Access

Implemented granular IAM permissions via AWS CDK to strictly control access to specific API endpoints:

new PolicyStatement({
  actions: ['execute-api:Invoke'],
  effect: Effect.ALLOW,
  resources: [`arn:aws:execute-api:${context.region}:${context.accountId}:${apiStack.httpApi.apiId}/*/POST/netsuite/*`],
});

Infrastructure Replicability & Robust Testing

Used AWS CDK to create replicable, environment-specific infrastructure, facilitating consistent deployments across development, staging, and production environments:

export const createStacks = async () => {
  const app = new App();
  const context = await getContext(app);

  new SecretStack(app, 'SecretsStack', { env: context.env });
};

Complex Workflow Management

Leveraged AWS Step Functions to effectively orchestrate complex workflows, such as order processing, refunds, and inventory adjustments:

const stepFunction = new stepfunctions.StateMachine(this, 'OrderProcessingStateMachine', {
  definition: workflowDefinition,
  stateMachineType: stepfunctions.StateMachineType.STANDARD,
});

Outcomes

The middleware solution provided real-time synchronisation, significantly enhancing operational efficiency and data accuracy. By adopting a secure, scalable AWS-based architecture, the integration has successfully streamlined business processes and provided a robust foundation for future digital innovation.

Project Images