Abstract:
In today’s world, real-time data or streaming data can be conceived as a continuous and changing sequence of
data that continuously arrive at a system to store or process. Big Data is also one of the hottest research topics in big data
computing and it requires different approaches: techniques, tools and architecture. Big data security also faces the need to
effectively enforce security policies to protect sensitive data. Trying to satisfy this need, we proposed the secure big data
pipeline architecture for the scalability and security. Throughout our work, we emphasize about the security of message. We
use Apache Kafka and Apache Storm for real time streaming pipeline and also use sticky policies and encryption/decryption
algorithm for security.
