Enterprise Framework

Software Solutions in the Enterprise

Pycharm Lambda Debug Error: docker.errors.APIError: 500 Server Error: Internal Server Error ("b'Mounts denied: \r\nThe path /Applications

Error


/usr/local/bin/sam local invoke HelloWorldFunction --template /Users/ramiscaray/Source/Learning/PyCharm/.aws-sam/build/template.yaml --event "/private/var/folders/l5/g1p_pxjx1sb8vcqmhkw77sr40000gn/T/[Local] HelloWorldFunction-event8.json" --debug-port 57029 --debugger-path "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev" --debug-args "-u /tmp/lambci_debug_files/pydevd.py --multiprocess --port 57029 --file"
Invoking app.lambda_handler (python3.7)

Fetching lambci/lambda:python3.7 Docker container image......
Mounting /Users/ramiscaray/Source/Learning/PyCharm/.aws-sam/build/HelloWorldFunction as /var/task:ro,delegated inside runtime container
Traceback (most recent call last):
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/docker/api/client.py", line 261, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.35/containers/6f2de107246c21d55146a3c737e94c7fcc1084af9dfb39b281f3839ca0195569/start

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/sam", line 33, in <module>
    sys.exit(load_entry_point('aws-sam-cli==0.53.0', 'console_scripts', 'sam')())
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/lib/telemetry/metrics.py", line 96, in wrapped
    raise exception  # pylint: disable=raising-bad-type
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/lib/telemetry/metrics.py", line 62, in wrapped
    return_value = func(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/commands/local/invoke/cli.py", line 86, in cli
    parameter_overrides,
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/commands/local/invoke/cli.py", line 151, in do_cli
    context.function_name, event=event_data, stdout=context.stdout, stderr=context.stderr
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/commands/local/lib/local_lambda.py", line 100, in invoke
    self.local_runtime.invoke(config, event, debug_context=self.debug_context, stdout=stdout, stderr=stderr)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/local/lambdafn/runtime.py", line 83, in invoke
    self._container_manager.run(container)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/local/docker/manager.py", line 95, in run
    container.start(input_data=input_data)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/samcli/local/docker/container.py", line 188, in start
    real_container.start()
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/docker/models/containers.py", line 400, in start
    return self.client.api.start(self.id, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/docker/api/container.py", line 1095, in start
    self._raise_for_status(res)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/docker/api/client.py", line 263, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/Cellar/aws-sam-cli/0.53.0/libexec/lib/python3.7/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 500 Server Error: Internal Server Error ("b'Mounts denied: \r\nThe path /Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev\r\nis not shared from OS X and is not known to Docker.\r\nYou can configure shared paths from Docker -> Preferences... -> File Sharing.\r\nSee https://docs.docker.com/docker-for-mac/osxfs/#namespaces for more info.\r\n.'")



Cause:  Docker does not have access to the /Applications path where PyCharm is.

Solution:

  • In Docker > Open Preferences
    • File Sharing
      • click (+)
      • Add /Applications
      • Click Apply & Restart


C# Version of Pipe

C# Version of Pipe in Angular RxJS 


Do you like the Pipe function in angular rsjx?   Want a C# version?  You can use IEnumerable.Aggregate

Below is an XUnit Test

using System;
using System.Linq;
using Xunit;

namespace PipeTest
{
    public class UnitTest1
    {
        [Fact]
        public void Test2()
        {
            Func<int, int>[] funcs = { AddOne, AddTwo, AddThree, AddFour };

            int initialValue = 1;

            int finalTotal = Pipe(initialValue, funcs);

            Assert.Equal(11, finalTotal);
        }

        public int Pipe(int initialValue, Func<int, int>[] funcs)
        {
            int finalTotal =
                funcs.Aggregate(initialValue,
                                    (accumulator, next) => next(accumulator),
                                    finalResult => {
                                        return finalResult;
                                    }
                                );

            return finalTotal;
        }

        public int AddOne(int value)
        {
            return value + 1;
        }

        public int AddTwo(int value)
        {
            return value + 2;
        }

        public int AddThree(int value)
        {
            return value + 3;
        }

        public int AddFour(int value)
        {
            return value + 4;
        }
    }
}

Setup a Nodejs AWS Lambda Layer with Custom Code and Setup VSCode Debugging

Steps to Success:

Last Updated - 2019-12-21

Steps Overview

  1. Create a Root Source Code Folder
  2. Add a nodejs folder to the Root Source Code Folder
  3. Initialize NPM in the nodejs folder
  4. Configure VSCode to run
  5. Add jest for unit testing
  6. Configure VSCode to Debug Jest Tests
  7. Create a custom JS code for use
  8. Create a AWS Lambda function to use the custom JS

Folder Structure

Our folder structure will end up looking like this:

/Source

    /my-layer

        /nodejs

            /src


Now that we know the overview, lets get started

1) Create a Root Source Code Folder called my-layer

$ mkdir my-layer

2) Create a nodejs folder under my-layer

$ cd my-layer && mkdir nodejs

3) Initial nodejs packages in the nodejs folder

$ cd nodejs
$ npm init -y

4) Install Jest as our testing framework

$ npm install --save-dev jest

5) Create a src folder for custom code to be placed

$ mkdir src && cd src


The structure of our new folders should be:

    /my-layer

        /nodejs

            /src

            package.json


Open Visual Studio Code and add the nodejs folder to the Workspace

  1. Open Visual Studio Code
  2. Open the folder /my-layer
  3. Choose File > Add Folder to Workspace
  4. Choose File > Save Workspace As
  5. Save the Workspace as /my-layer/my-layer-workspace.code-workspace


Open /my-layer/nodejs/package.json and update the scripts and jest section so it has the following:

{
"name": "directory-layer",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"build": "rm ../directory-layer.zip && npm install && zip -r ../helloworld-layer.zip ../nodejs/",
"test": "jest"
},
"keywords": [],
"author": "",
"license": "ISC",
"dependencies": {
"jest-cli": "^24.9.0",
"moment": "^2.24.0"
},
"devDependencies": {
"jest": "^24.9.0"
},
"jest": {
"testEnvironment": "node",
"verbose": true
}
}


Create a two new files under /src

  • helloworld.js 
  • helloworld.text.js


In helloworld.js, add the following text

function myFunction() {
return "hello";
}

module.exports = myFunction;


in helloworld.text.js, add the following text

const directory = require('./directory');

var result = directory();

test('test helloworld', () => {
expect(result).toBe('hello');
});


Run the Jest Test to verify it works

$ npm test


Run the npm build command to generate the helloworld-layer.zip file.  We'll upload the helloworld-layer.zip file to AWS

$ cd nodejs
$ npm run build


The following file should have been created:

/my-layer/helloworld-layer.zip


Upload the new layer to AWS Lambda > Layers

  • Create Layer
    • Name:  my-layer
    • Choose > Upload a .zip file
    • Click Upload
    • Choose my-layer.zip

Create a test function to use the new my-layer layer

  • Create Function
    • Choose > Author from scratch
    • Function name:  test-lambda-layer
    • Runtime:  Node.js 10.x

Update function index.js file with the following source code:

const helloworld = require('/opt/nodejs/src/helloworld');

const result = helloworld();

console.log(result);

exports.handler = async (event) => {
// TODO implement
const response = {
statusCode: 200,
body: JSON.stringify(`Hello from Lambda! ${result}`),
};
return response;
};

Add Visual Studio Launch Config for Jest Debugging.  You can follow these directions here at https://github.com/microsoft/vscode-recipes/tree/master/debugging-jest-tests

or you can do the following:

1) Click the Debug and Run icon from the left Navigation Bar.

2) Click the Gear icon next to "Debug and Run"

3) Paste the launch.json text.  


{
"version": "0.2.0",
"configurations": [
{
"type": "node",
"request": "launch",
"name": "Jest All",
"program": "${workspaceFolder}/node_modules/.bin/jest",
"args": [
"${workspaceFolder}",
"--runInBand"
],
"console": "integratedTerminal",
"internalConsoleOptions": "neverOpen",
"disableOptimisticBPs": true
},
{
"type": "node",
"request": "launch",
"name": "Jest Current File",
"program": "${workspaceFolder}/node_modules/.bin/jest",
"args": [
"${file}",
"--config",
"jest.config.js"
],
"console": "integratedTerminal",
"internalConsoleOptions": "neverOpen",
"disableOptimisticBPs": true
}
]
}

4) You can then press the > start debugging icon.







CloudFormation SAM Template: Give AWS::Serverless::Function role to access DynamoDB


DoSomethingFunction:
Type: AWS::Serverless::Function
Description: Do Something
Properties:
FunctionName: 'DoSomething'
Policies:
- AmazonDynamoDBFullAccess
Handler: 'do_something.lambda_handler'
Runtime: !Ref LambdaRunTimeVersion
MemorySize: !Ref LambdaMemorySize
Timeout: !Ref LambdaTimeout
Events:
MySQSEvent:
Type: SQS
Properties:
Queue: !GetAtt SqsQueue.Arn
BatchSize: 10
VpcConfig:
SecurityGroupIds: !Ref VpcSecurityGroupIds
SubnetIds: !Ref VpcSubnetIds
Environment:
Variables:
'dynamodb_table_name': !Ref DynamoDBTable
'sqs_url': !Ref SqsQueue
Tags:
Name: 'do_something.lambda_handler'
Role: !GetAtt LambdaExecutionRole.Arn

LambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- sts:AssumeRole
Policies:
- PolicyName: allowLambdaLogs
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- logs:*
Resource: arn:aws:logs:*:*:*
- PolicyName: AllowDynamoDb
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- dynamodb:*
Resource: !GetAtt DynamoDBTable.Arn
- PolicyName: AllowSqs
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- sqs:ReceiveMessage
- sqs:DeleteMessage
- sqs:GetQueueAttributes
- sqs:ChangeMessageVisibility
Resource: !GetAtt SqsQueue.Arn

React Boilerplate - Add State Property to Container

Adding New State Property setting constants, actions, selectors, reducer, index

  • Container/Dashboard/constants.js
    • // Add constant for CHANGE_TAB
      export const CHANGE_TAB = 'boilerplate/Dashboard/CHANGE_TAB';
  • Container/Dashboard/actions.js
    • import { CHANGE_TAB } from './constants';
      
      /**
       * Changes the selected tab of the dashboard
       *
       * @param  {selectedTab} name The new text of the input field
       *
       * @return {object}    An action object with a type of CHANGE_SENSOR
       */
      export function changeTab(selectedTab) {
        const action = {
          type: CHANGE_TAB,
          selectedTab,
        };
        console.log(`changeTab(${selectedTab})`, action);
        return action;
      }
      
  • Container/Dashboard/selectors.js
    • // Create a selector that will get selectedTab from State
      const makeSelectSelectedTab = () =>
        createSelector(selectDashboard, dashboardState => 
      dashboardState.get('selectedTab'));
      
      // Add makeSelectSelectedTab to export
      export { makeSelectSelectedTab };
      
  • Container/Dashboard/reducer.js
    • import { CHANGE_TAB } from './constants';
      
      // The initial state of the App
      export const initialState = fromJS({
        selectedTab: 0
      });
      
      // Add CHANGE_TAB to reducer
      function dashboardReducer(state = initialState, action) {
        switch (action.type) {
          case CHANGE_TAB:
            // Set selected tab
            return state.set('selectedTab', action.selectedTab);
          default:
           return state;
        }
      }
      
      
  • Container/Dashboard/index.js
    • import { changeTab } from './actions';
      import { makeSelectSelectedTab } from './selectors';
      
      class Dashboard extends React.PureComponent {
        handleTabChange = (e, f) => {
          this.props.onChangeTab(f); 
        }
        render() {
          return (
            <div>hello</div>
          );
        }
      }
      
      // Add selectedTab, and onChangeTab to propTypes
      DashboardPage.propTypes = {
        loading: PropTypes.bool,
        error: PropTypes.oneOfType([PropTypes.object, PropTypes.bool]),
        repos: PropTypes.oneOfType([PropTypes.array, PropTypes.bool]),
        onSubmitForm: PropTypes.func,
        selectedTab: PropTypes.number,
        onChangeTab: PropTypes.func,
      };
       
      // Add onChangeTab method to props, it will be dispatch on calling.
      export function mapDispatchToProps(dispatch) {
        return {
          onChangeTab: selectedTab => {
            console.log(`onChangeTab(${selectedTab})`);
            dispatch(changeTab(selectedTab));
          }
        };
      }
      
      // Add selectedTab to props.  It retrieves from selector from state
      const mapStateToProps = (state, ownProps) => {
        return createStructuredSelector({
            selectedTab: makeSelectSelectedTab(),
            loading: makeSelectLoading(),
            error: makeSelectError(),
          }
        );
      };
       
      
  • Save and review

Google Cloud switch between multiple accounts project in App Engine

View gcloud configuration:

$ gcloud config list

You should see something like:

[core]
account = your.account@gmail.com
disable_usage_reporting = True
project = myproject-somename-123467

Your active configuration is: [default]

Unset the current cloud project

$ gcloud config unset project

Set the new project.  Your current App Engine name is something like:  my-other-project.appspot.com
Just use the first part without the appspot.com

$ gcloud config set project my-other-project

Run gcloud app deploy now

$ gcloud app deploy

You should see it now say something like

Services to deploy:

descriptor:      [/Users/yourname/Source/Repos/appengine/app.yaml]
source:          [/Users/yourname/Source/Repos/appengine]
target project:  [my-other-project]
target service:  [default]
target version:  [20181218t202435]
target url:      [https://my-other-project.appspot.com]


Do you want to continue (Y/n)?  y

sls deploy error - sh: sls: command not found

Running "sls deploy" generates error.  sh: sls: command not found after running 

sls is a part of the serverless framework.  You can install it by going to:

https://www.npmjs.com/package/serverless

or run the following

npm install -g serverless


Splunk Forwarder Linux Setup

Splunk Forwarder Linux Setup

                      # Get splunk wget -O splunkforwarder-7.2.1-be11b2c46e23-Linux-x86_64.tgz 'https://www.splunk.com/bin/splunk/DownloadActivityServlet?architecture=x86_64&platform=linux&version=7.2.1&product=universalforwarder&filename=splunkforwarder-7.2.1-be11b2c46e23-Linux-x86_64.tgz&wget=true' sudo tar -xvzf splunkforwarder-7.2.1-be11b2c46e23-Linux-x86_64.tgz -C /opt cd /opt/splunkforwarder/bin # Start splunk forwarder sudo ./splunk start --accept-license sudo ./splunk enable boot-start # Change the default password from changeme sudo ./splunk edit user admin -password newp@$$w0rd -role admin -auth admin:changeme # Add forward server (Where to send data) sudo ./splunk add forward-server YourSplunkServer.com:9997 -auth admin:newp@$$w0rd # Point to directory to monitor sudo ./splunk add monitor /opt/log/www1/ Reference: https://www.splunk.com/en_us/resources/videos/splunk-education-getting-data-in-with-forwarders.html

Brew Update Error: The following directories are not writable by your user:

Error: The following directories are not writable by your user:

/usr/local/include

/usr/local/share/man/man3

/usr/local/share/man/man5

/usr/local/share/man/man7

/usr/local/share/man/man8

You should change the ownership of these directories to your user.

To resolve try:

sudo chown -R $(whoami) /usr/local/include /usr/local/share/man/man3 /usr/local/share/man/man5 /usr/local/share/man/man7 /usr/local/share/man/man8

If that fails, just reinstall brew by going to https://brew.sh/

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"


Swagger Codegen for PHP

First install Java 7 or 8

1 - Set environment variables for java

export JAVA_HOME=$(/usr/libexec/java_home)
or
export JAVA_HOME=`/usr/libexec/java_home -v 1.8`

export PATH=${JAVA_HOME}/bin:$PATH

2 - Clone the REPO 

$ git clone https://github.com/swagger-api/swagger-codegen.git

3 - Switch into the directory 

$ cd swagger-codegen

3 - Build the project form code using maven.  Direction for maven on other post.

$ mvn clean package

4 - Generate the client for your endpoint

Replace the below item in < > with your custom values.

java -jar modules/swagger-codegen-cli/target/swagger-codegen-cli.jar generate \
   -i ../acumatica_client/swagger/swagger_customerschema_17.200.001.json \
   -l php \
   --invoker-package "<RootNamespace>\<CustomNamespace>" \
   --git-user-id "<yourgithubuserid>" \
   --git-repo-id "<yourgithubrepo>" \
   -o ~/Source/Repos/<my_output_folder>

5.  Switch to the output folder from above <my_output_folder>

cd ~/Source/Repos/<my_output_folder>

6.  Make git_push executable so we can push to source control.

chmod +x git_push.sh

7.  Push to source control.

./git_push.sh

7a - git pull origin master --allow-unrelated-histories


8 - Open the newly generated README.md file, copy the text from the Installation & Usage, Composer.json section 

~/Source/Repos/<my_output_folder>/SwaggerClient-php/README.md

9 - Open your PHP client project that you want to use the swagger generated code in and paste the previously copied text from README.md

{
  "repositories": [
    {
      "type": "git",
      "url": "https://github.com/<yourgithubuserid>/<yourgithubrepo>.git"
    }
  ],
  "require": {
    "<yourgithubuserid>/<yourgithubrepo>": "*@dev"
  }
}

11 - Run Composer Install from the PHP Client project to install references

$ composer install

Note you may need to run 

$ composer update --lock

Add a generic called runme.php file and add code from README.md

categorySchemaDeleteById($id);
} catch (Exception $e) {
    echo 'Exception when calling CategorySchemaApi->categorySchemaDeleteById: ', $e->getMessage(), PHP_EOL;
}

?>

and run it
   
$ php runme.php