|
One Gas
Tulsa, OK
Senior Hadoop Developer, Sep 2012 – Present
·
Installed and
configured HadoopMapReduce, HDFS, Developed multiple MapReduce jobs in java,
R and Python for data cleaning and preprocessing.
·
Configured and tested Hadoop for a non-distributed mode, as a single
Java process using Hadoop 0.21.0 and Java 1.6
·
Configured and tested Hadoop for pseudo-distributed mode where each
Hadoop job runs in a separate Java process
·
Worked in
CloudEra environment with desktop virtualization using VMWare Player to
provision the Hadoop jobs in department cluster
·
Importing and exporting
data into HDFS and Hive using Sqoop
·
Experienced in
defining job flows
·
Experienced in
managing and reviewing Hadoop log files
·
Supported Map
Reduce Programs those are running on the cluster
·
Involved in
loading data from UNIX file system to HDFS.
·
Installed and
configured Hive and documented Hive UDFs.
·
Involved in
loading data from UNIX file system to HDFS.
·
Installed and
configured Hive and documented Hive UDFs.
·
Involved in
creating Hive tables, loading with data and writing hive queries which will
run internally in map way.
·
Collaborate in
creating Hive tables, loading with data and writing hive queries which will
run internally in map way.
·
Data Analysis
using Regression models in R and Python.
Environment:
Hadoop, MapReduce, HDFS, Hive, Java (jdk1.6), Hadoop distribution of
Hortonworks, Cloudera, MapR, DataStax, IBM DataStage 8.1(Designer,
Director, Administrator), Flat files, Oracle 11g/10g, PL/SQL, SQL*PLUS,
Toad 9.6, Windows NT, UNIX Shell Scripting, Autosys r11.0.
Mercedes
Montville, NJ
Hadoop
Developer, Feb 2011 – Aug 2012
·
Installed and
configured HadoopMapReduce, HDFS, Developed multiple MapReduce jobs in
java, R and Pyhton for data cleaning and processing.
·
Administered a Hadoop cluster with 20 physical machines with around
100 virtual nodes and an HDFS file system with 5 TB capacity.
·
Automated all
the jobs, for pulling data from FTP server to load data into Hive tables,
using Oozie workflows.
·
Performed
shell based administration of workspace for students and peers by assigning
permissions using DFSShell command line interface and DFS Admin command set
for HDFS access
·
Extensively
worked with Apache platforms like HDFS, HBase and PIG for large data set
analysis
·
Design process
implementation of HRMS SaaS solution
·
Designed and
implemented MapReduce algorithms in Java for clustering problems using byte
streams to minimize memory usage.
·
Involved in
creating Hive tables, loading with data and writing hive queries which will
run internally in map way
·
Responsible
for ETL design, development, testing and review of code along with
supporting documents like Unit test case and technical handover documents.
·
Responsible
for integration testing and User acceptance testing.
·
Developing
complex ETL mapping and its corresponding sessions & worklets,
workflows.
·
End-to-end
testing of Data warehouse/Data Mart load.
·
Responsible
for understanding the scope of the project and requirement gathering.
Environment:
Hadoop, MapReduce, HDFS, Hive, Java (jdk1.6), Hadoop distribution of
Hortonworks, Cloudera, MapR, DataStax, IBM DataStage 8.1(Designer,
Director, Administrator), Flat files, Oracle 11g/10g, PL/SQL, SQL*PLUS,
Toad 9.6, Windows NT, UNIX Shell Scripting, Autosys r11.0.
Yankee
Candle South Deerfield, MA
Hadoop Administrator, July 2009-Jan 2011
·
Supported
code/design analysis, strategy development and project planning.
·
Developed
multiple MapReduce jobs in Java for data cleaning and preprocessing.
·
Assisted with
data capacity planning and node forecasting.
·
Analyzed the
requirements from the customers and participated in Agile
·
Data extracted
from Teradata and pushing into Hadoop using Sqoop.
·
T logs are
pulled form log server and stored in the FTP server hourly
wise; this data is pushed into Hadoop by deleting data in the FTP server.
·
Knowledge on
Java Virtual Machines (JVM) and multithreaded processing.
·
Developed PIG
Latin scripts to extract the data from the web server output files to load
into HDFS
·
Developed the
PIG UDF’S to pre-process the data for analysis
·
Develop HIVE
queries for the analysts
·
Developed
workflow in Oozie to automate the tasks of loading the data into HDFS
and pre-processing with PIG
·
Responsible to
manage data coming from different sources
· Supported Map Reduce Programs those are running on the
cluster
· Transactional Data Analysis using R and Python
Environment:
Hadoop,
MapReduce, HDFS, Hive, Java (jdk1.6), Hadoop distribution of Hortonworks,
Cloudera, MapR, DataStax, Spring 2.5, Hibernate 3.0, JSF, Servlets , JDBC,
JSP,JSTL, JPA, JavaScript, Eclipse 3.4, log4j,Oracle 10g, CVS, CSS, Xml,
XSLT, SMTP, Windows-XP.
Bhartiya
Technologies, India
Java/J2EE
Consultant, Oct 2007-April 2009
·
Involved in
various phases of Software Development Life Cycle.
·
Used Eclipse
6.0 as IDE for application development.
·
Validated all
forms using Struts validation framework and implemented Tiles framework in
the presentation layer.
·
Configured
Struts framework to implement MVC design patterns
·
Designed and
developed GUI using JSP, HTML, DHTML and CSS.
·
Worked with
JMS for messaging interface.
·
Used Hibernate
for handling database transactions and persisting objects.
·
Deployed the
entire project on WebLogic application server.
·
Used AJAX for
interactive user operations and client side validations.
·
Used XML for
ORM mapping relations with the java classes and the database.
·
Used XSL
transforms on certain XML data.
·
Developed ANT
script for compiling and deployment.
·
Performed unit
testing using JUnit.
·
Extensively
used log4j for logging the log files.
·
Used
Subversion as the version control system
·
Data Cleaning
and Analysis using R.
Environment:
Java/J2EE, Oracle 10g, SQL, PL/SQL, JSP, EJB , Struts, Hibernate,
WebLogic 8.0, HTML, AJAX, Java Script, JDBC, XML,UML, JUnit, log4j,
MyEclipse 6.0.
Hierarchy
Technologies, India
Java/J2EE Developer, Aug 2006 –
Sep 2007
·
Involved in
analyzing the client requirements and convert them into technical
specifications
·
Worked in
Analysis, Design and Coding for client developmentusingJ2EE stack using
Eclipse platform.
·
Involved
in creating web-based java components like client Applets and client side
UI using JFC in Eclipse
·
Developed
PL/SQL stored procedures to perform complex database operations.
·
Designed and
developed SQL queries in the application
·
Developed Design
documents for various components identified in the system.
·
Generated the Hibernate
XML and Java Mappings for the schemas
·
Used Rational
Application Developer (RAD) as Integrated Development Environment
(IDE).
·
Extensively
used Core Java, Servlets, JSP and XML.
·
Used Struts
1.2 in presentation tier.
·
Generated the Hibernate
XML and Java Mappings for the schemas.
·
Used Subversion
as the version control system
·
Involved in
various phases of Software Development Life Cycle.
·
Created UML Diagrams
(Class and Sequence) during Design Phase using Visio.
·
Deployed the
entire project on WebLogic application server.
·
Transactional
Data Analysis in R and Matlab.
Environment:
Java/J2EE, Oracle 10g, SQL, PL/SQL, JSP, EJB ,
Struts, Hibernate, WebLogic 8.0, HTML, AJAX, Java Script, JDBC, XML,
JMS, XSLT, UML, JUnit, log4j, MyEclipse 6.0.
|