AWS CLI is a command line tool used for interacting with AWS services.

In this post, I'll share the command to sync your local files to S3 bucket by removing files in the bucket that are not present in the local folder.

$ aws s3 sync . s3://bucket-name --delete

--delete option …

AWS Cognito provides authentication, authorization, and user management for your webapps. In this article I will show Angular snippets to perform authentication with AWS Cognito credentials.

First, I'll show the CognitoService class with just signIn functionality. I've removed other operations like register, confirm registration, etc. from the snippet for simplicity …

AWS Lambda is a compute service that lets you run code without provisioning or managing servers. In this article I will show how to create and manage simple AWS Lambda function written in Python.

Create files and folders as shown below. Technically, app.py is sufficient to create a function …

To update your GitHub forked repository to that of the remote using TortoiseGit, follow the below listed steps:

• Add upstream repository that has to be synced with your fork

TortoiseGit -> Settings -> Remote

• Fetch the upstream repository

TortoiseGit -> Fetch

• Pull the upstream repository

TortoiseGit -> Pull

• Resolve any merge conflicts

• Push to …

To load a DataFrame from a MySQL table in PySpark

source_df = sqlContext.read.format('jdbc').options(
          url='jdbc:mysql://localhost/database_name',
          driver='com.mysql.jdbc.Driver',
          dbtable='SourceTableName',
          user='your_user_name',
          password='your_password').load()

And to write a DataFrame to a MySQL table

destination_df.write.format('jdbc').options(
          url='jdbc:mysql …

To find the difference between the current row value and the previous row value in spark programming with PySpark is as below

Let say, we have the following DataFrame and we shall now calculate the difference of values between consecutive rows.

+---+-----+
| id|value|
+---+-----+
|  1|   65|
|  2|   66|
|  3|   65|
|  4 …

Amazon Kinesis is a fully managed stream hosted on AWS. It is used to collect and process large streams of data in real time. Along with Kinesis Analytics, Kinesis Firehose, AWS Lambda, AWS S3, AWS EMR you can build a robust distributed application to power your real-time monitoring dashboards, do …

Amazon DynamoDB is a NoSQL database service hosted on AWS. It is a fully managed and scalable document store database. It is quite similar to MongoDB.

As the data grows, scan operation on full table would return parts of the data with the LastEvaluatedKey. The application should initiate scan again …

Configuring and managing multiple SSH keys for GIT accounts, in Ubuntu is shown below:

Using ssh-keygen create a public/private key and name it as user1_github, user1_bitbucket

create a file ~/.ssh/config and put the following in it.

# user1 github account
Host user1_github
HostName github.com
PreferredAuthentications publickey
IdentityFile ~/.ssh …

With timeseries data we often require to resample on different intervel to feed in to our analytics model.

Pandas resample have a built-in list of widely used methods. However, if the built-in methods are not sufficient, it is always possible to write a custom function to resample.

This post shows …