Python-kinesis
Python에서 Kinesis Datastream에 연동하여 데이터를 주고/받는 과정
The process of sending and receiving data by connecting to Kinesis Datastream in Python
Kinesis로 데이터를 전달할 때, byte혹은 bytearray로 전달 해야 함 (encode('utf-8')로 전달 해 주어야 함)
When sending data to Kinesis, it must be passed as byte or bytearray (You need to pass it using encode('utf-8'))
- Requirements
- Install boto3, kinesis
- Requirements
- Install boto3, kinesis
pip3 install -U pip
python3 -m pip install boto3 kinesis
-
Setting Credential and config
아래 두개의 파일을 생성한다.
credentials, config 두개의 파일을 ~/.aws에 위치시킨다
~/.aws/
- credentials —> setting access_key, sercret_key
-
Setting Credential and config
Create the following two files.
Place the credentials and config files in ~/.aws
~/.aws/
- credentials —> setting access_key, secret_key
[default]
aws_access_key_id=
aws_secret_access_key=
- config —> setting region, type
- config —> setting region, type
[default]
region=region_info
output=json
- Test setting
- test
- Test setting
- test
import boto3
from kinesis.producer import KinesisProducer
from kinesis.consumer import KinesisConsumer
consumer = KinesisConsumer(stream_name=env.KINESIS_STREAM_NAME)
for message in consumer:
print("############")
print(message)
print("############")
- test consumer
- test consumer
from kinesis.consumer import KinesisConsumer
from kinesis.state import DynamoDB
consumer = KinesisConsumer(stream_name='my-stream', state=DynamoDB(table_name='my-kinesis-state'))
for message in consumer:
print ("Received message: {0}".format(message))
- test producer
- test producer
from kinesis.producer import KinesisProducer
producer = KinesisProducer(stream_name='my-stream')
producer.put('Hello World from Python')