Apache Spark

Steps to Connect HiveServer2 using Apache Spark JDBC Driver and Python

Apache Spark supports both local and remote metastore. You can connect to remote HiveServer2 using Apache Spark JDBC drivers. Hive JDBC driver for Spark2 is available in the jars folder located in the spark installation directory. In this post, we will check steps to connect HiveServer2 using Apache Spark JDBC Driver and Python. Steps to Connect HiveServer2 using Apache Spark JDBC Driver and Python There are various methods that you can use to connect to HiveServer2. Using Spark JDBC driver is one of easy method. Methods to Access Hive Tables…

Continue ReadingSteps to Connect HiveServer2 using Apache Spark JDBC Driver and Python
Comments Off on Steps to Connect HiveServer2 using Apache Spark JDBC Driver and Python
November 2, 2018
General

Set and Use Environment Variable inside Python Script

It is somewhat difficult when it comes to setting and using bash environment variables in python script file. The same step is very easy and straight forward using shell script. In this post, we will check one of the method to set and use environment variable inside python scrip file. Note that, steps mentioned in this post helps only if you are setting and using that variable inside same process i.e. in same python script. There is no way you can modify bash script from python and use that variable…

Continue ReadingSet and Use Environment Variable inside Python Script
Comments Off on Set and Use Environment Variable inside Python Script
October 24, 2018
BigData

Steps to Connect HiveServer2 from Python using Hive JDBC Drivers

HiveServer2 has a JDBC driver and It supports both embedded and remote access to HiveServer2. Usually, remote HiveServer2 is recommended for production environment as it does not require direct metastore or HDFS access to be given to Hive users. In this article, we will check steps to Connect HiveServer2 from Python using Hive JDBC Drivers. Steps to Connect HiveServer2 from Python using Hive JDBC Drivers Hive JDBC driver is one of the widely used method to connect to HiveServer2. You can use the Hive JDBC with Python Jaydebeapi open source module.…

Continue ReadingSteps to Connect HiveServer2 from Python using Hive JDBC Drivers
Comments Off on Steps to Connect HiveServer2 from Python using Hive JDBC Drivers
September 21, 2018
BigData

Execute Hive Beeline JDBC String Command from Python

To perform any analysis, you need to have data in place. To collect data, you may have to connect your application to different data source. In this article, we will discuss on one of such approach to execute Hive Beeline JDBC string command from Python application. This is one of the simple and easy approach to connect to Kerberos HiveServer2 using Beeline shell. I was working on one of the machine learning project to predict query execution time on Hadoop Hive cluster. We were gathering various features from the HiveQL…

Continue ReadingExecute Hive Beeline JDBC String Command from Python
Comments Off on Execute Hive Beeline JDBC String Command from Python
September 4, 2018
BigData

Step by Step Guide Connecting HiveServer2 using Python Pyhive

Data plays important role in every decision-making process. You may have to connect to various remote servers to get required data for your application. This article explains how to connect Hive running on remote host (HiveSever2) using commonly used Python package, Pyhive. In this article, we will check step by step guide Connecting HiveServer2 using Python Pyhive. There are lot of other Python packages available to connect to remote Hive, but Pyhive package is one of the easy and well-maintained and supported package. There is a option to connect to…

Continue ReadingStep by Step Guide Connecting HiveServer2 using Python Pyhive
2 Comments
May 4, 2018
Netezza

Connecting Netezza using Python pyodbc – Working Example

In this modern era, data plays very important role in every decision making. Most of applications that is written in various programming languages require access to relational databases to get required data for its process. Before you can run SQL statements to create, update, delete, or retrieve data, you must connect to a database. In this article, we will check process and working example on connecting Netezza using Python pyodbc driver. Netezza also supports JDBC driver: How to Connect Netezza using JDBC Driver and working Examples Netezza ODBC drivers Before attempting…

Continue ReadingConnecting Netezza using Python pyodbc – Working Example
Comments Off on Connecting Netezza using Python pyodbc – Working Example
March 28, 2018
General

Python File Word Count using Dictionary

Dictionaries are one of the best data types introduced in the Python. The dictionary holds data in form on Key:value pair. In this article, will present you the solution to Python File Word Count using Dictionary. Text File Acquire the text file from which you want to count repetition of each and every word. For the testing purpose, create any file with some of your favourite story or anything. Python File Word Count using Dictionary Let's work step by step on building this game. In this program, we are going…

Continue ReadingPython File Word Count using Dictionary
Comments Off on Python File Word Count using Dictionary
March 29, 2017