In part 3 section, I would like to introduce how to create Amazon Kinesis Data Firehose. Twitter data can be loaded to your environment by applying Amazon Kinesis Data Firehose.
1. Create S3 bucket
Please create raw, entities and sentiment bucket by referring to my previous article or other document.
2. Create IAM Role for Amazon Kinesis Data Firehose
Create Amazon Kinesis Data Firehose
Firstly, open Kinesis console in the AWS management console.
You can see Get started screen if it is your first creation.
Select Kinesis Data Firehose and click Create Delivery stream button.
Input Data stream name as you like.
You can change number of open shards in this section. Input appropriate shards number according to your performance requirement.
In my test environment, I selected default value, 1 and click Create data stream button.
Select Direct PUT as Source. And select Amazon S3 as Destination.
Then, input Delivery stream name.
You can see Destination settings screen after selecting Amazon S3 as destination.
Input your S3 bucket information and S3 bucket prefix.
As one of example, I input raw/ as S3 bucket prefix.
Then, click Create Delivery stream button.
After creation, you can see your Amazon Data Kinesis Firehose resource for raw/ prefix as below.
Please repeat previous procedure and create Amazon Data Kinesis Firehose resource for sentiment/ and entities/ prefix as same.
That’s all for Amazon Kinesis Data Firehose creation. I hope you could create it with ease.
From next section, I would like to introduce how to create function to get data by using Twitter API.
Basically, I followed the procedure provided by AWS webpage. Please refer to below URL if necessary, though it is written in Japanese…