SUMMARY:

·           More than seven years of comprehensive IT experience in Big Data domain with tools like Hadoop, Hive and other open source tools/technologies in Banking, Healthcare and Energy. 

·           Substantial experience writing MapReduce jobs in Java, PIG, Flume, Zookeeper and Hive and Storm.

·           Experience in development of Big Data projects using Hadoop, Hive, HDP, PIG, Flume, Storm and MapReduce open source tools/technologies.

·           Extensive Knowledge on automation tools such as Puppet and Chef.

·           Hands on experience in installing, configuring and using ecosystem components like Hadoop MapReduce, HDFS, Hbase, AVRO, Zoo Keeper, Oozie, Hive, HDP, Cassandra, Sqoop, PIG, Flume. 

·           Extensive experience in SQL and NoSQL development. 

·           In-depth understanding of Data Structure and Algorithms. 

·           Experience in web-based languages such as HTML, CSS, PHP, XML and other web methodologies including Web Services and SOAP. 

·           Extensive experience in all the phases of the software development lifecycle (SDLC).

·           Experience in deploying applications in heterogeneous Application Servers TOMCAT, Web Logic and Oracle Application. Server. 

·           Extensive knowledge of NoSQL databases such as HBase.

·           Worked on Multi Clustered environment and setting up Cloudera Hadoop echo System. 

·           Background with traditional databases such as Oracle, Teradata, Netezza, SQL Server, ETL tools / processes and data warehousing architectures. 

 

EDUCATION:

BS in Computer Science UPTU, Lucknow, India 2007

 

PROFESSIONAL EXPERIENCE:

 

Sr. Big Data/Hadoop Architect

 

Responsibilities:

·           Delivered Working Widget Software using EXTJS4, HTML5, RESTFUL Web services, JSON Store, Linux, Hadoop, ZOOKEEPER, NO SQL databases, JAVA, SPRING Security, JBOSS Application Server for Big Data analytics.

·           Developed a custom AVRO Framework capable of solving small files problem in Hadoop and also extended PIG and Hive tools to work with it. 

·           Working on use cases, data requirements and business value for implementing a Big Data Analytics platform.

·           Working on configuring and Maintaining Hadoop environment on AWS.

·           Developed application component interacting with MongoDB.

·           Working on Modifying Chef Recipes used to configure the Hadoop stack.

·           Evaluate Puppet framework and tools to automate the cloud deployment and operations.

·           Working on Installing and configuring Hive, HDP, PIG, Sqoop, Flume, Storm and Oozie on the Hadoop cluster.

·           Working in analyzing data using Hive, PIG, Storm and custom MapReduce programs in Java.

·           Developed Use cases and Technical prototyping for implementing PIG, HDP, HIVE and HBASE.

·           Working as a Big Data/Hadoop Architect on Integration and Analytics based on Hadoop, SOLR and web Methods technologies.

·           Working in implementing Hadoop with the AWS EC2 system using a few instances in gathering and analyzing data log files.

·           Working on data using Sqoop from HDFS to Relational Database Systems and vice-versa.

·           Working on GE on loading files to HIVE and HDFS from MongoDB.

·           Founded and developed environmental search engine engine using PHP5, JAVA, Lucene/SOLR, Apache and MYSQL.

·           Led the evaluation of Big Data software like Splunk, Hadoop for augmenting the warehouse, identified use cases and led Big Data Analytics solution development for Customer Insights and Customer Engagement teams.

·           Worked on Distributed/Cloud Computing (MapReduce/Hadoop, PIG, HBase, AVRO, Zookeeper, etc.), Amazon Web Services (S3, EC2, EMR, etc.), Oracle SQL Performance Tuning and ETL, Java 2 Enterprise and Web Development.

·           Designed techniques and wrote effective and successful programs in JAVA, Linux shell scripting to push the large data including the Text and Byte type of data to successfully migrate to NO SQL Stores using various Data Parser techniques in addition to Map Reduce jobs.  

·           Involved in scheduling Oozie workflow engine to run multiple Hive and PIG jobs.

·           Worked on TOAD for Data Analysis, ETL/Informatica for data mapping and the data transformation between the source and the target database. 

·           Working on Hive/Hbase vs RDBMS, imported data to Hive, HDP created tables, partitions, indexes, views, queries and reports for BI data analysis.

·           Developing data pipeline using Flume, Sqoop, PIG and Java map reduce to ingest customer behavioral data and financial histories into HDFS for analysis.

·           Currently working on XML parsing using PIG, Hive, HDP and Redshift.

·           Working on architected solutions that process massive amounts of data on corporate and AWS cloud based servers.

·           Uses Splunk to detect any malicious activity against webservers 

·           Tuned the Hadoop Clusters and Monitored for the memory management and for the Map Reduce jobs, to enable healthy operation of Map Reduce jobs to push the data from SQL to No SQL store.

·           Working in importing streaming logs and aggregating the data to HDFS through Flume.

 

Big Data/Hadoop Developer/Architect

 

Responsibilities:

·           Worked on NoSQL databases including Hbase and MongoDB.

·           Used data stores included Accumulo/Hadoop and graph database.

·           Exploited Hadoop MySQL–Connector to store Map Reduce results in RDBMS.

·           Used Dell Crowbar as a wrapper to Chef to deploy Hadoop.

·           Collected data from different databases (i.e. Teradata, Oracle, MySQL) to Hadoop

·           Used Oozie and Zookeeper for workflow scheduling and monitoring.

·           Worked on Designing and Developing ETL Workflows using Java for processing data in HDFS/Hbase using Oozie.

·           Created Hbase tables to store various data formats of PII data coming from different portfolios

·           Conduct vulnerability analyses; reviewing, analyzing and correlating threat data from available sources such as Splunk.  

·           Designed, planned and delivered proof of concept and business function/division based implementation of Big Data roadmap and strategy project (Apache Hadoop stack with Tableau) in JP Morgan Chase using Hadoop.

·           Developed MapReduce jobs in Java for data cleaning and preprocessing. 

·           Importing and exporting data into HDFS and Hive using Sqoop.

·           Used Bash Shell Scripting, Sqoop, AVRO, Hive, HDP, Redshift, PIG and Java Map/Reduce daily to develop ETL, batch processing, and data storage functionality.   

·           Responsible for developing data pipeline using flume, Sqoop and PIG to extract the data from weblogs and store in HDFS.

·           Working on extracting files from MongoDB through Sqoop and placed in HDFS and processed.

·           Worked on Hadoop installation & configuration of multiple nodes on AWS EC2 system.

·           Experienced in running Hadoop streaming jobs to process terabytes of XML format data. 

·           Cluster coordination services through Zoo Keeper. 

·           Involved in loading data from UNIX file system to HDFS 

·           Installed and configured Hive and also written Hive UDFs. 

·           Worked on setting up PIG, Hive, Redshift and Hbase on multiple nodes and developed using PIG, Hive, Hbase, MapReduce and Storm. 

·           Installed and configured Hadoop through Amazon Web Services in cloud. 

·           Design and implement data processing using AWS Data Pipeline.

·           Drove holistic tech transformation to Big Data platform for JP Morgan Chase, create strategy, define blueprint, design roadmap, build end-to-end stack, evaluate leading technology options, benchmark selected products, migrate products, reconstruct information architecture, introduce metadata management, leverage machine learning, productionize consolidated data store: Hadoop, MR, Hive, HDP and MapReduce.

·           Developed Simple to complex MapReduce Jobs using Hive and PIG.

·           Worked on automate monitoring and optimizing large volume data transfer processes between Hadoop clusters and AWS.  

·           Strong knowledge on Data Warehousing experience using Informatica Power Center.

·           Configure and manage Splunk Forwarders, Splunk Indexers and Splunk Search Heads. 

·           Automated all the jobs for pulling data from FTP server to load data into Hive tables using Oozie workflows. 

 

Hadoop Developer

 

Responsibilities:

·           Worked on analyzing Hadoop cluster and different Big Data analytic tools including PIG, Hbase database and Sqoop. 

·           Responsible for building scalable distributed data solutions using Hadoop. 

·           Built Analytics KPI engine using Python and PIG. 

·           Worked on the data to import into AWS instance and Mapreduce jobs will be executed to analyze the data.

·           Developed fielded search dataset using Hadoop and Accumulo.

·           Working with HP to scale their existing data pipeline to handle 10x the data to match our growth trajectory.

·           Worked in Big Data Analytics Initiative from system design to live operation in production.

·           Created customizations report using pentaho and MongoDB/ Cassandra provisioning/ decommission.

·           Installed Oozie workflow engine to run multiple Hive, Redshift and PIG Jobs.

·           Production and pre-production cloud infrastructure based on AWS cloud.

·           Worked on supporting and managing Chef Server.

·           Worked on Hadoop AVRO Files to Network File System for recording Audit data.

·           Worked on supporting MapReduce Programs those are running on the cluster

·           Help assisting finding bugs and learning Tableau Desktop for Splunk.

·           Created HBase tables to store variable data formats of PII data coming from different portfolios. 

·           Implemented a script to transmit sysprin information from Oracle to Hbase using Sqoop. 

·           Implemented best income logic using PIG scripts and UDFs. 

·           Worked on tuning the performance PIG queries. 

·           Involved in loading data from UNIX file system to HDFS. 

·           Worked on Apache web log data into Hadoop, transform it into a standard AVRO format, and output it to a Snappy compressed AVRO file.

·           Installed and Configured AWS Data Pipeline to run multiple AWS.

·           Expertise in developing and deploying Splunk, installing Splunk forwarders and developing dashboards. 

·           Responsible for cluster maintenance, adding and removing cluster nodes, cluster monitoring and troubleshooting, manage and review data backups, manage and review Hadoop log files. 

·           Built firmware check-in service using MongoDB, Java/Spring as both hosted and cloud service.

·           Develop reusable tools to efficiently handle the AWS migration pipeline, mostly scripts.

·           Installed Oozie workflow engine to run multiple Hive, HDP, Redshift and PIG jobs. 

·            Supported in setting up QA environment and updating configurations for implementing scripts with PIG and Sqoop.

 

JAVA Developer

 

Responsibilities:

·           Database analyzing, design and implementation.

·           Worked on entire data pipeline for automating using Flume and for the jobs scheduled periodically using Oozie.

·           Used JavaScript for client side validation. 

·           Database connections and code implementation.

·           Used Python because supports multiple programming paradigms, including object-oriented, imperative and functional programming styles.

·           Expertise in writing shell scripts and Oozie workflows

·           Worked on developing and extending serialization frameworks like AVRO.

·           Development and maintenance of splunk dashboards based on the requirements. 

·           Worked on configure AWS EMR (Elastic MapReduce).

·           Written the Apache PIG scripts to process the HDFS data.

·           Reviewing the existing Hadoop environment and make recommendations of new features that may be available and performance tuning with the other tools like Hive, PIG, MapReduce, Storm and Flume.

·           Worked on Monitoring, Replication and Sharding Techniques in MongoDB.

·           Performed Manual Testing, reported defects in JIRA and was responsible to keep track of them.

·           Used BI solution as a custom application build using OBIEE as front end and ODI and PL/SQL used for ETL. Developed HTML and JSP pages using Struts.

·           Designed GUI Components using Tiles frame work and Validation frame work.

·           Worked in Installation, Configuration and Management of Hadoop Cluster spanning multiple racks using automated tools like puppet and chef.

·           Worked on providing support for AWS Data Pipeline.

·           Worked on automating the jobs using Oozie in the project.

·           Used sequence and AVRO file formats and snappy compressions while storing data in HDFS.

·           Developed MapReduce application using Hadoop, Redshift, MapReduce programming and Hbase.

·           Monitoring Hadoop scripts which take the input from HDFS and load the data into Hive.