Enterprise Framework

Software Solutions in the Enterprise

git clone repository from Windows to AWS Code Commit error, Permission denied, fatal: Could not read from remote repository.

git clone repository from Windows to AWS Code Commit error, Permission denied, fatal: Could not read from remote repository.. 


Example Error Command:

$ git clone ssh://git-codecommit.us-east-1.amazonaws.com/v1/repos/my-code-repository

Error:

Cloning into 'my-code-repository'...
ramis@git-codecommit.us-east-1.amazonaws.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Solution Prerequisites:

  • SSH Key created for Windows and in ~/.ssh folder
  • SSH Public Key has been uploaded in AWS > IAM > User > Robert > Security Credentials > SSH keys for AWS CodeCommit

Solution:

You have to specify the SSH-KEYID from AWS IAM Account in the git clone

$ git clone ssh://{SSH-KEYID}@git-codecommit.us-east-1.amazonaws.com/v1/repos/{RepositoryName}

Example:

$ git clone ssh://BZRSTYKLXIY6X4RUOKYL@git-codecommit.us-east-1.amazonaws.com/v1/repos/my-code-repository

dotnet CLI for Entity Framework Core Tools

To get "dotnet ef" you have to install Entity Framework Core Tools


Reference

https://docs.microsoft.com/en-us/ef/core/cli/dotnet

Install: 


$ dotnet tool install --global dotnet-ef

$ cat << \EOF >> ~/.zprofile

# Add .NET Core SDK tools

export PATH="$PATH:/Users/ramiscaray/.dotnet/tools"

EOF

# Make available to current session
$ zsh -l

$ dotnet add package Microsoft.EntityFrameworkCore.Design

$ dotnet ef

$ dotnet ef migrations add AddBookDatabase

$ dotnet ef database update




.NET 5 is .NET Core renamed

.NET 5 is what should have been .NET Core 4.x.  Microsoft instead renamed it for two reason according to an article they posted.

"We skipped version numbers 4.x to avoid confusion with .NET Framework 4.x.

We dropped "Core" from the name to emphasize that this is the main implementation of .NET going forward. .NET 5.0 supports more types of apps and more platforms than .NET Core or .NET Framework.

ASP.NET Core 5.0 is based on .NET 5.0 but retains the name "Core" to avoid confusing it with ASP.NET MVC 5. Likewise, Entity Framework Core 5.0 retains the name "Core" to avoid confusing it with Entity Framework 5 and 6."




VSCode Override Prettier Settings for JavaScript and JSON to use single Quote

You can override VS Code pretty settings by adding a .prettierrc file to the root of your project to override javascript double quote with a single quote.


NOTE:  Prettier extension for VSCode is required to do this.

The below configuration will set .js extension files to use a single quote. 

{
"singleQuote": false,
"overrides": [
{
"files": ["**/*.js"],
"options": {
"singleQuote": true
}
}
]
}



CloudFormation CodeBuildProject (AWS::CodeBuild::Project) Mount EFS (Elastic File System) in a VPC with Subnets

How To:  CloudFormation CodeBuild Project (AWS::CodeBuild::Project) To Mount EFS (Elastic File System) in a VPC with Subnets

These are the steps needed to mount CodeBuild Project to EFS by setting the FileSystemLocation property and referencing it in BuildSpec.yaml


In this example below, EFS is Mounted to 3 different Subnets in 3 different AZ's


  FileSystemResource:
    Type: 'AWS::EFS::FileSystem'
    Properties:
      BackupPolicy:
        Status: ENABLED
      PerformanceMode: maxIO
      ...
      ...
      ...

  MountTargetResource1:
    Type: AWS::EFS::MountTarget
    Properties:
      FileSystemId: !Ref FileSystemResource
      SubnetId: subnet-4f2e0e48
      SecurityGroups:
      - !GetAtt MountTargetVPC.DefaultSecurityGroup

  MountTargetResource2:
    Type: AWS::EFS::MountTarget
    Properties:
      FileSystemId: !Ref FileSystemResource
      SubnetId: subnet-620be258
      SecurityGroups:
      - !GetAtt MountTargetVPC.DefaultSecurityGroup

  MountTargetResource3:
    Type: AWS::EFS::MountTarget
    Properties:
      FileSystemId: !Ref FileSystemResource
      SubnetId: subnet-d133f80d
      SecurityGroups:
      - !GetAtt MountTargetVPC.DefaultSecurityGroup


NOTE: If your EFS is Mounted to 1 or more Subnets in different AZ's, you will need to have the same Subnets mapped in AWS::CodeBuild::Project.  This applies to EC2 instances as well, create EFS Mount Targets for each of your EC2 Instance Subnet Locations.

The AWS::CodeBuild::Project example below has 3 Subnets that are the same as the EFS Mounted networks show above.

Project:
  Type: AWS::CodeBuild::Project
  Properties:
    Name: myProjectName
    Description: A description about my project
    ServiceRole: !GetAtt ServiceRole.Arn
    Artifacts:
      Type: no_artifacts
    Environment:
      Type: LINUX_CONTAINER
      ComputeType: BUILD_GENERAL1_SMALL
      Image: aws/codebuild/java:openjdk-8
      EnvironmentVariables:
      - Name: varName
        Type: varType
        Value: varValue
    Source:
      Location: codebuild-demo-test/0123ab9a371ebf0187b0fe5614fbb72c
      Type: S3
    TimeoutInMinutes: 10
    Tags:
      - Key: Key1
        Value: Value1
      - Key: Key2
        Value: Value2
    VpcConfig
      SecurityGroupIds: 
          - sg-1f23ed4a29c64cccb
      Subnets: 
          - subnet-4f2e0e48
          - subnet-620be258
          - subnet-d133f80d
      VpcId: vpc-0fdf3d2a773deef1d
    FileSystemLocation:
      # Identifier will create a environment variable that you can reference in BuildSpec to reference the mounted EFS path
      #     This is usually the EFS File System Id without a Hyphen
      #     NOTE:  this can be lower case and referenced like $CODEBUILD_fs9325925
      Identifier: fs9325925

      # Location Format:  {efs-file-system-id}.efs.{region}.amazonaws.com:{path}
      Location: fs-9325925.efs.us-east-1.amazonaws.com:/    

      # MountOptions: String    

      # Local folder to mount to efs to, example:   /efs )
      MountPoint: /efs

      # Type:  Valid Values:   EFS 
      Type: EFS


This is the BuildSpec example to reference the Mounted EFS.  A environment variable called CODEBUILD_fs9325925 has been created to allow you access the folder that has been mounted to EFS.

In the below example, $CODEBUILD_fs9325925 = /efs

version: 0.2
phases:
  build:
    commands:
      - printenv #Output set environment variables
      - cp index.html $CODEBUILD_fs9325925





Pycharm Lambda Debug Error: docker.errors.APIError: 500 Server Error: Internal Server Error ("b'Mounts denied: \r\nThe path /Applications

Error


/usr/local/bin/sam local invoke HelloWorldFunction --template /Users/ramiscaray/Source/Learning/PyCharm/.aws-sam/build/template.yaml --event "/private/var/folders/l5/g1p_pxjx1sb8vcqmhkw77sr40000gn/T/[Local] HelloWorldFunction-event8.json" --debug-port 57029 --debugger-path "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev" --debug-args "-u /tmp/lambci_debug_files/pydevd.py --multiprocess --port 57029 --file"
Invoking app.lambda_handler (python3.7)

Fetching lambci/lambda:python3.7 Docker container image......
Mounting /Users/ramiscaray/Source/Learning/PyCharm/.aws-sam/build/HelloWorldFunction as /var/task:ro,delegated inside runtime container
Traceback (most recent call last):
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/docker/api/client.py", line 261, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.35/containers/6f2de107246c21d55146a3c737e94c7fcc1084af9dfb39b281f3839ca0195569/start

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/sam", line 33, in <module>
    sys.exit(load_entry_point('aws-sam-cli==0.53.0', 'console_scripts', 'sam')())
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/lib/telemetry/metrics.py", line 96, in wrapped
    raise exception  # pylint: disable=raising-bad-type
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/lib/telemetry/metrics.py", line 62, in wrapped
    return_value = func(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/commands/local/invoke/cli.py", line 86, in cli
    parameter_overrides,
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/commands/local/invoke/cli.py", line 151, in do_cli
    context.function_name, event=event_data, stdout=context.stdout, stderr=context.stderr
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/commands/local/lib/local_lambda.py", line 100, in invoke
    self.local_runtime.invoke(config, event, debug_context=self.debug_context, stdout=stdout, stderr=stderr)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/local/lambdafn/runtime.py", line 83, in invoke
    self._container_manager.run(container)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/local/docker/manager.py", line 95, in run
    container.start(input_data=input_data)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/local/docker/container.py", line 188, in start
    real_container.start()
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/docker/models/containers.py", line 400, in start
    return self.client.api.start(self.id, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/docker/api/container.py", line 1095, in start
    self._raise_for_status(res)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/docker/api/client.py", line 263, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 500 Server Error: Internal Server Error ("b'Mounts denied: \r\nThe path /Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev\r\nis not shared from OS X and is not known to Docker.\r\nYou can configure shared paths from Docker -> Preferences... -> File Sharing.\r\nSee https://docs.docker.com/docker-for-mac/osxfs/#namespaces for more info.\r\n.'")



Cause:  Docker does not have access to the /Applications path where PyCharm is.

Solution:

  • In Docker > Open Preferences
    • File Sharing
      • click (+)
      • Add /Applications
      • Click Apply & Restart


C# Version of Pipe

C# Version of Pipe in Angular RxJS 


Do you like the Pipe function in angular rsjx?   Want a C# version?  You can use IEnumerable.Aggregate

Below is an XUnit Test

using System;
using System.Linq;
using Xunit;

namespace PipeTest
{
    public class UnitTest1
    {
        [Fact]
        public void Test2()
        {
            Func<int, int>[] funcs = { AddOne, AddTwo, AddThree, AddFour };

            int initialValue = 1;

            int finalTotal = Pipe(initialValue, funcs);

            Assert.Equal(11, finalTotal);
        }

        public int Pipe(int initialValue, Func<int, int>[] funcs)
        {
            int finalTotal =
                funcs.Aggregate(initialValue,
                                    (accumulator, next) => next(accumulator),
                                    finalResult => {
                                        return finalResult;
                                    }
                                );

            return finalTotal;
        }

        public int AddOne(int value)
        {
            return value + 1;
        }

        public int AddTwo(int value)
        {
            return value + 2;
        }

        public int AddThree(int value)
        {
            return value + 3;
        }

        public int AddFour(int value)
        {
            return value + 4;
        }
    }
}

Setup a Nodejs AWS Lambda Layer with Custom Code and Setup VSCode Debugging

Steps to Success:

Last Updated - 2019-12-21

Steps Overview

  1. Create a Root Source Code Folder
  2. Add a nodejs folder to the Root Source Code Folder
  3. Initialize NPM in the nodejs folder
  4. Configure VSCode to run
  5. Add jest for unit testing
  6. Configure VSCode to Debug Jest Tests
  7. Create a custom JS code for use
  8. Create a AWS Lambda function to use the custom JS

Folder Structure

Our folder structure will end up looking like this:

/Source

    /my-layer

        /nodejs

            /src


Now that we know the overview, lets get started

1) Create a Root Source Code Folder called my-layer

$ mkdir my-layer

2) Create a nodejs folder under my-layer

$ cd my-layer && mkdir nodejs

3) Initial nodejs packages in the nodejs folder

$ cd nodejs
$ npm init -y

4) Install Jest as our testing framework

$ npm install --save-dev jest

5) Create a src folder for custom code to be placed

$ mkdir src && cd src


The structure of our new folders should be:

    /my-layer

        /nodejs

            /src

            package.json


Open Visual Studio Code and add the nodejs folder to the Workspace

  1. Open Visual Studio Code
  2. Open the folder /my-layer
  3. Choose File > Add Folder to Workspace
  4. Choose File > Save Workspace As
  5. Save the Workspace as /my-layer/my-layer-workspace.code-workspace


Open /my-layer/nodejs/package.json and update the scripts and jest section so it has the following:

{
"name": "directory-layer",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"build": "rm ../directory-layer.zip && npm install && zip -r ../helloworld-layer.zip ../nodejs/",
"test": "jest"
},
"keywords": [],
"author": "",
"license": "ISC",
"dependencies": {
"jest-cli": "^24.9.0",
"moment": "^2.24.0"
},
"devDependencies": {
"jest": "^24.9.0"
},
"jest": {
"testEnvironment": "node",
"verbose": true
}
}


Create a two new files under /src

  • helloworld.js 
  • helloworld.text.js


In helloworld.js, add the following text

function myFunction() {
return "hello";
}

module.exports = myFunction;


in helloworld.text.js, add the following text

const directory = require('./directory');

var result = directory();

test('test helloworld', () => {
expect(result).toBe('hello');
});


Run the Jest Test to verify it works

$ npm test


Run the npm build command to generate the helloworld-layer.zip file.  We'll upload the helloworld-layer.zip file to AWS

$ cd nodejs
$ npm run build


The following file should have been created:

/my-layer/helloworld-layer.zip


Upload the new layer to AWS Lambda > Layers

  • Create Layer
    • Name:  my-layer
    • Choose > Upload a .zip file
    • Click Upload
    • Choose my-layer.zip

Create a test function to use the new my-layer layer

  • Create Function
    • Choose > Author from scratch
    • Function name:  test-lambda-layer
    • Runtime:  Node.js 10.x

Update function index.js file with the following source code:

const helloworld = require('/opt/nodejs/src/helloworld');

const result = helloworld();

console.log(result);

exports.handler = async (event) => {
// TODO implement
const response = {
statusCode: 200,
body: JSON.stringify(`Hello from Lambda! ${result}`),
};
return response;
};

Add Visual Studio Launch Config for Jest Debugging.  You can follow these directions here at https://github.com/microsoft/vscode-recipes/tree/master/debugging-jest-tests

or you can do the following:

1) Click the Debug and Run icon from the left Navigation Bar.

2) Click the Gear icon next to "Debug and Run"

3) Paste the launch.json text.  


{
"version": "0.2.0",
"configurations": [
{
"type": "node",
"request": "launch",
"name": "Jest All",
"program": "${workspaceFolder}/node_modules/.bin/jest",
"args": [
"${workspaceFolder}",
"--runInBand"
],
"console": "integratedTerminal",
"internalConsoleOptions": "neverOpen",
"disableOptimisticBPs": true
},
{
"type": "node",
"request": "launch",
"name": "Jest Current File",
"program": "${workspaceFolder}/node_modules/.bin/jest",
"args": [
"${file}",
"--config",
"jest.config.js"
],
"console": "integratedTerminal",
"internalConsoleOptions": "neverOpen",
"disableOptimisticBPs": true
}
]
}

4) You can then press the > start debugging icon.







CloudFormation SAM Template: Give AWS::Serverless::Function role to access DynamoDB


DoSomethingFunction:
Type: AWS::Serverless::Function
Description: Do Something
Properties:
FunctionName: 'DoSomething'
Policies:
- AmazonDynamoDBFullAccess
Handler: 'do_something.lambda_handler'
Runtime: !Ref LambdaRunTimeVersion
MemorySize: !Ref LambdaMemorySize
Timeout: !Ref LambdaTimeout
Events:
MySQSEvent:
Type: SQS
Properties:
Queue: !GetAtt SqsQueue.Arn
BatchSize: 10
VpcConfig:
SecurityGroupIds: !Ref VpcSecurityGroupIds
SubnetIds: !Ref VpcSubnetIds
Environment:
Variables:
'dynamodb_table_name': !Ref DynamoDBTable
'sqs_url': !Ref SqsQueue
Tags:
Name: 'do_something.lambda_handler'
Role: !GetAtt LambdaExecutionRole.Arn

LambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- sts:AssumeRole
Policies:
- PolicyName: allowLambdaLogs
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- logs:*
Resource: arn:aws:logs:*:*:*
- PolicyName: AllowDynamoDb
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- dynamodb:*
Resource: !GetAtt DynamoDBTable.Arn
- PolicyName: AllowSqs
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- sqs:ReceiveMessage
- sqs:DeleteMessage
- sqs:GetQueueAttributes
- sqs:ChangeMessageVisibility
Resource: !GetAtt SqsQueue.Arn

React Boilerplate - Add State Property to Container

Adding New State Property setting constants, actions, selectors, reducer, index

  • Container/Dashboard/constants.js
    • // Add constant for CHANGE_TAB
      export const CHANGE_TAB = 'boilerplate/Dashboard/CHANGE_TAB';
  • Container/Dashboard/actions.js
    • import { CHANGE_TAB } from './constants';
      
      /**
       * Changes the selected tab of the dashboard
       *
       * @param  {selectedTab} name The new text of the input field
       *
       * @return {object}    An action object with a type of CHANGE_SENSOR
       */
      export function changeTab(selectedTab) {
        const action = {
          type: CHANGE_TAB,
          selectedTab,
        };
        console.log(`changeTab(${selectedTab})`, action);
        return action;
      }
      
  • Container/Dashboard/selectors.js
    • // Create a selector that will get selectedTab from State
      const makeSelectSelectedTab = () =>
        createSelector(selectDashboard, dashboardState => 
      dashboardState.get('selectedTab'));
      
      // Add makeSelectSelectedTab to export
      export { makeSelectSelectedTab };
      
  • Container/Dashboard/reducer.js
    • import { CHANGE_TAB } from './constants';
      
      // The initial state of the App
      export const initialState = fromJS({
        selectedTab: 0
      });
      
      // Add CHANGE_TAB to reducer
      function dashboardReducer(state = initialState, action) {
        switch (action.type) {
          case CHANGE_TAB:
            // Set selected tab
            return state.set('selectedTab', action.selectedTab);
          default:
           return state;
        }
      }
      
      
  • Container/Dashboard/index.js
    • import { changeTab } from './actions';
      import { makeSelectSelectedTab } from './selectors';
      
      class Dashboard extends React.PureComponent {
        handleTabChange = (e, f) => {
          this.props.onChangeTab(f); 
        }
        render() {
          return (
            <div>hello</div>
          );
        }
      }
      
      // Add selectedTab, and onChangeTab to propTypes
      DashboardPage.propTypes = {
        loading: PropTypes.bool,
        error: PropTypes.oneOfType([PropTypes.object, PropTypes.bool]),
        repos: PropTypes.oneOfType([PropTypes.array, PropTypes.bool]),
        onSubmitForm: PropTypes.func,
        selectedTab: PropTypes.number,
        onChangeTab: PropTypes.func,
      };
       
      // Add onChangeTab method to props, it will be dispatch on calling.
      export function mapDispatchToProps(dispatch) {
        return {
          onChangeTab: selectedTab => {
            console.log(`onChangeTab(${selectedTab})`);
            dispatch(changeTab(selectedTab));
          }
        };
      }
      
      // Add selectedTab to props.  It retrieves from selector from state
      const mapStateToProps = (state, ownProps) => {
        return createStructuredSelector({
            selectedTab: makeSelectSelectedTab(),
            loading: makeSelectLoading(),
            error: makeSelectError(),
          }
        );
      };
       
      
  • Save and review