[AWS] Twitter Analysis ~Part7: Create a node.js application in EC2 instance~

Purpose
In part 7 section, I would like to explain how to create a node.js application in Cloud9 EC2 environment. Let me show you the scope of this section as below.
Prerequisite
Create a node.js application in Cloud9 environment
Open Cloud9 console and click Your environments button.

You can see your environment created in the previous part. Click Open IDE button.

You can see IDE screen as below. Click File and New File.

And create a node.js application as below.

My sample application extracts data from twitter. Twitter search result can be collected from S3 bucket.
Let me share my sample application customized from AWS sample code.

twitter_reader.js
—————————————————————————————————
var Twit = require(‘twit’)
const AWS = require(‘aws-sdk’);
const Twitter = require(‘twitter-lite’);

////////////// Environment Setting//////////////////
const twitterUserName = “XXXX”; // XXXX: Twitter User Name
const region = “ap-northeast-1”; 
const deliveryStreamName = “twitter-analysis-raw”;
const secretName = “twitterAPI-secrets”;
////////////////////////////////////////////////////

const firehose = new AWS.Firehose({apiVersion: ‘2015-08-04’, region: region });
const secretsManager = new AWS.SecretsManager({ region: region });

async function getSecrets() {
const data = await secretsManager.getSecretValue({SecretId: secretName}).promise();
return JSON.parse(data.SecretString);
}

async function putRecordToFirehose(tweet) {

const recordParams = {
DeliveryStreamName: deliveryStreamName,
Record: {
Data: JSON.stringify(tweet) +’\n’
}
};
await firehose.putRecord(recordParams).promise();
}

async function main() {

const secrets = await getSecrets();
const twitter = new Twitter({
consumer_key: secrets.ConsumerApiKey,
consumer_secret: secrets.ConsumerApiSecret,
access_token_key: secrets.AccessToken,
access_token_secret: secrets.AccessTokenSecret
});
//console.log(“Start twitter”)
//console.log(twitter)
var stream = twitter.stream(‘statuses/filter’, { track: ‘YYYYY’ });  //YYYY: Search Word
stream.on(‘data’, function(tw) {
var text = tw.text;
var user_name = tw.user.name;

})
.on(“start”, _ => console.log(“start”))
.on(“data”, putRecordToFirehose)
.on(“error”, error => console.log(“error”, error))
.on(“end”, _ => console.log(“end”));
;
}
main();
————————————————————————————-

Next, open terminal in IDE screen and switch to root  user.
Then, change permission to be executed as script as below.

Then execute npm init command for initialization.

And install aws-sdk by executing npm command.

Next, install twitter-lite library.

Then, define service to execute the script as daemon as below.

And enable service by executing systemctl command.

Finally start your defined service as below.

After a few minutes, please check if extracted data is stored in the S3 bucket.

OK, it looks fine !!

That’s all for this topic. In my next article, let me show you how to create Lambda function to transfer data to Amazon Comprehend service. I would be glad if you read it continuously.

Reference
Basically, I followed the procedure provided by AWS webpage. Please refer to below URL if necessary, though it is written in Japanese…
If you think this article is beneficial for you, I would be glad if you click below icon for my motivation.
ブログランキング・にほんブログ村へ

コメント