![]() ![]() sudo cp /usr/local/Cellar/hive/3.1.3/libexec/conf/hive-site.xml. Now, copy hive-site.xml file to the Spark conf directory. To edit nf, give the following command: vi nfĪnd add the below line: =hive Now, specify Hive configuration in Spark default conf file. cd $HIVE_HOME/scripts/metastore/upgrade/mysql/ Logging in into MySQL mysql> use metastore mysql> source hive-schema-3.1.0.mysql.sql mysql> quit Įxit MySQL and run below command in terminal schematool -dbType mysql -initSchema -dryRun Running Hive $ hive hive > show tables hive > exit Integrating Hive with SparkĬopy Spark default configuration to Spark conf directory cd /usr/local/Cellar/apache-spark/3.2.0/libexec/conf sudo cp nf Manually create the tables using the script available at the following location. You need to create tables in MySQL which are required for the metastore. Once you have updated the path in hive-site.xml, then save the file using :wq. Go inside hive/warehouse using below command and copy the path cd hive/warehouse pwdĮdit the hive-site.xml file and update the path of /Users//hive/warehouse location of default database for the warehouse Note: Download hive-site.xml file from here, and then run the following commands: cd ~/Downloads sudo cp hive-site.xml /usr/local/Cellar/hive/3.1.3/libexec/conf chmod -R 777 /tmp/hiveĬreate a directory hive/warehouse cd ~ mkdir -p hive/warehouse Key in followed by to save the bash_profile file.Īfter saving the above file, execute the following command: source ~/.bash_profile Download JDBC connector cd ~/Downloads tar xzf mysql-connector-java-8.0.29.tar sudo cp mysql-connector-java-8.0.29/mysql-connector-java-8.0.29.jar /usr/local/Cellar/hive/3.1.3/libexec/lib/ cd /usr/local/Cellar/hive/3.1.3/libexec/conf export HADOOP_HOME=/usr/local/Cellar/hadoop/3.3.3/libexec export HIVE_HOME=/usr/local/Cellar/hive/3.1.3/libexec Use the key to change vi editor’s mode to “Insert” mode. If installation is successful then please follow the below steps -Įdit the bash_profile file and add the following environment variables - vi ~/.bash_profile This is a known issue, Please refer to this article to resolve the problem. Note: You may get a “No bottle available!” error if you are running a Macbook pro having an M1 chip. Update Brew and Install Hive - brew update brew install hive Setting up MySQL database $ mysql -u root -p mysql> CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 mysql> CREATE USER IDENTIFIED BY ‘password’ mysql> GRANT ALL on metastore.* TO mysql> FLUSH PRIVILEGES mysql> quit Installing Hive You will need to type the root user password after you run this command, and once you are done you should see a mysql prompt. Now, you can connect to the MySQL server using the command: mysql -u root -p ![]() Since we used the command brew services start mysql to start MySQL, your macOS will start it automatically, after each reboot. To locate the installation path and other details of Spark, enter the following command: brew info apache-spark If it is not installed, then to install it, open a terminal and execute the below command - brew install apache-spark Now that you have brew installed, this installs the latest version of Spark by default. Now we have installed Java 8, we can check the installation with the following command. Or on newer Mac OS version(s): brew install - cask adoptopenjdk8 Let’s install OpenJDK using brew: brew cask install adoptopenjdk8 In case, Java is not installed then Installing OpenJDK 8 Note: If you already have Java installed, please don’t execute the below commands. Open a terminal window and execute the following command: /bin/bash -c “$(curl -fsSL )" Check Java Version java -version run the current cell and insert a new cell below it.to run the command in the current cell and move to the next cell below it.Using shortcuts in both command and edit modes: ![]()
0 Comments
Leave a Reply. |