H2O is an open source, in-memory, distributed, fast, and scalable machine learning and predictive analytics platform that allows you to build machine learning models on big data. H2O data refers to the data that you intend to use to build machine learning models in H2O. Before you can build models, you first need to import and activate your data in H2O.
Some key things to know about activating data in H2O:
- H2O accepts data in various file formats like CSV, SVMLite, ARFF, XLS, and others.
- The data must be imported and parsed into H2O before you can build models.
- H2O stores the data in-memory in a distributed fashion for fast model building.
- The data is split into training, validation, and test sets before modeling.
- Activating data in H2O prepares it for supervised and unsupervised machine learning.
Now let’s look at the various ways to import and activate data in H2O step-by-step.
Uploading a Data File from Your Local Machine
If the data file resides on your local machine, you can directly upload it to H2O using the intuitive web UI or the h2o.importFile() command in R or Python.
Here are the steps to upload a local CSV file named iris.csv:
Using the H2O Web UI
- Launch the H2O Flow web interface by typing http://localhost:54321 in your browser.
- Click on the “Upload File” option at the top right.
- Select the iris.csv file on your local machine and click “Open.”
- Choose the correct column types and options like header, separator, etc.
- Click the “Import” button to import the file into H2O.
Using h2o.importFile() in R
- Launch H2O and initialize an h2o object.
- Use h2o.importFile() and pass the local file path.
- Specify the destination frame, column types, header, separator, etc.
- The data will be imported and activated in H2O.
“`r
library(h2o)
h2o.init()
iris_path <- "iris.csv" iris_h2o <- h2o.importFile(path = iris_path, destination_frame = "iris_h2o", col.types = c("numeric", "numeric", "numeric","factor")) ``` Using h2o.importFile() in Python
- Import h2o and initialize the h2o cluster.
- Use h2o.import_file() and pass the local file path.
- Specify the destination frame, column types, header, separator, etc.
- The data will be imported and activated in H2O.
“`python
import h2o
h2o.init()
iris_path = “iris.csv”
iris_h2o = h2o.import_file(path = iris_path,
destination_frame = “iris_h2o”,
col_types = [“numeric”, “numeric”, “numeric”, “enum”])
“`
Uploading a local file allows you to activate data in H2O with just a few lines of code. Make sure the path is correct and options like header, separator, column types, etc. are specified correctly for accurate imports.
Importing Data from URLs
Instead of a local file, you can also directly import data into H2O from a URL via HTTP or HTTPS. Here are the steps:
Using the H2O Web UI
- Launch the H2O Flow web interface.
- Click on “Import Files” and choose the “From URL” tab.
- Enter the URL of the data file.
- Specify parsing options like column types, header, separator, etc.
- Click the “Get” button to fetch the file.
- Click “Import” to import the parsed data into H2O.
Using h2o.importFile() in R
- Initialize H2O.
- Use h2o.importFile() with the ‘path’ parameter set as URL.
- Specify parsing options.
- Data will be imported into H2O from the URL.
“`r
h2o.init()
url <- "https://raw.githubusercontent.com/h2oai/h2o/master/smalldata/jira/pub-180.csv" jira_data <- h2o.importFile(url, destination_frame="jira_data") ``` Using h2o.import_file() in Python
- Initialize H2O.
- Use h2o.import_file() with the ‘path’ parameter set as URL.
- Specify parsing options.
- Data will be imported into H2O from the URL.
“`python
import h2o
h2o.init()
url = “https://raw.githubusercontent.com/h2oai/h2o/master/smalldata/jira/pub-180.csv”
jira_data = h2o.import_file(path = url, destination_frame=”jira_data”)
“`
Using a URL allows you to import data directly into H2O from anywhere without first downloading the file. Ensure the URL is accessible and valid.
Importing Data from S3 Buckets
To import big data files residing in AWS S3 buckets, specify an S3 URL with the bucket path prefixed by “s3n://” in h2o.importFile() or h2o.import_file().
Here is an example:
Using h2o.importFile() in R
“`r
h2o.init()
s3_path <- "s3n://bucket/path/to/file.csv" s3_data <- h2o.importFile(path = s3_path, destination_frame = "s3_data") ``` Using h2o.import_file() in Python
“`python
h2o.init()
s3_path = “s3n://bucket/path/to/file.csv”
s3_data = h2o.import_file(path = s3_path,
destination_frame=”s3_data”)
“`
The key steps are:
- Specify s3 URL with s3n:// prefix
- Provide authentication credentials
- Set destination frame
- Data will be imported into H2O from S3 bucket
This provides a fast and efficient way to activate big data for machine learning in H2O.
Importing Data from HDFS
To import data from HDFS storage, prefix the HDFS path with “hdfs://” in h2o.importFile() or h2o.import_file().
Here is an example:
Using h2o.importFile() in R
“`r
h2o.init()
hdfs_path <- "hdfs://namenode/path/to/file.csv" hdfs_data <- h2o.importFile(path = hdfs_path, destination_frame = "hdfs_data") ``` Using h2o.import_file() in Python
“`python
h2o.init()
hdfs_path = “hdfs://namenode/path/to/file.csv”
hdfs_data = h2o.import_file(path = hdfs_path,
destination_frame = “hdfs_data”)
“`
The key steps are:
- Specify hdfs URL with hdfs:// prefix
- Provide authentication credentials if secure
- Set destination frame
- Data will be imported into H2O from HDFS
This enables building models in H2O from big data stored in HDFS.
Importing Data from Hive
To import data directly from Hive tables, use the “import_sql_table” command in h2o Python package.
Here is an example:
“`python
h2o.init()
table_name = “hive_table_name”
hive_data = h2o.import_sql_table(connection_url=”jdbc:hive2://localhost:10000″, table=table_name,
username=”username”, password=”password”)
“`
The key steps are:
- Specify Hive JDBC URL
- Provide authentication credentials
- Set Hive table name
- Data will be imported into H2O from Hive table
This provides a convenient way to access and model Hive data using H2O.
Importing Data from JDBC Sources
You can import data into H2O from any JDBC compatible relational database like MySQL, Postgres, Oracle, etc.
Here is an example using PostgreSQL:
Using h2o.importSqlTable() in R
“`r
h2o.init()
jdbc_url <- "jdbc:postgresql://localhost/test" sql_query <- "SELECT * FROM table_name" pg_data <- h2o.importSqlTable(jdbc_url, sql_query, username = "username", password = "password") ``` Using h2o.import_sql_table() in Python
“`python
h2o.init()
jdbc_url = “jdbc:postgresql://localhost/test”
sql_query = “SELECT * FROM table_name”
pg_data = h2o.import_sql_table(jdbc_url, sql_query,
username=”username”,
password=”password”)
“`
The key steps are:
- Set JDBC connection URL
- Provide SQL query
- Set username and password
- Data is imported into H2O from SQL database
This enables accessing relational data for machine learning with H2O.
Importing Data from Spark RDDs and DataFrames
To import data from Spark:
Using as_h2o_frame() (Sparkling Water)
- Initialize H2OContext from pysparkling or sparkling-water.
- Convert Spark RDD or DataFrame to H2OFrame.
- Data is imported into memory in H2O.
“`python
from pysparkling import *
hc = H2OContext()
rdd = sc.parallelize([[1,2,3]])
h2o_df = hc.as_h2o_frame(rdd)
“`
“`scala
import org.apache.spark.h2o._
val h2oContext = H2OContext.getOrCreate()
val df = spark.read. … // Create Spark DataFrame
val h2o_frame = h2oContext.asH2OFrame(df)
“`
Using h2o.from_pandas() (Sparkling Water)
- Create Pandas DataFrame.
- Call h2o.from_pandas() to convert to H2OFrame.
- Data is imported into H2O.
“`python
import pandas as pd
df = pd.DataFrame(…)
import h2o
h2o_df = h2o.from_pandas(df)
“`
This enables converting Spark data to H2OFrames for machine learning.
Best Practices for Activating Data in H2O
Here are some best practices to follow while importing data into H2O:
- Ensure data is cleaned before importing into H2O.
- Specify column names, types, separator, header, etc. correctly.
- Handle missing values appropriately before importing data.
- Split data into train, validation, test sets before modeling.
- Use a destination frame name instead of default frame name.
- Preprocess data if needed before feeding into models.
- Encode categorical variables before importing into H2O.
- Check for balanced classes in case of supervised learning.
- Confirm data was imported correctly by inspecting in H2O Flow.
Following these best practices will ensure your data is activated properly in H2O for machine learning.
Conclusion
In summary, H2O provides many options to import data from local files, URLs, S3, HDFS, JDBC sources, Spark, and other systems into memory for fast and scalable machine learning. Some key steps are:
- Specify source and destination details correctly during import.
- Parse data accurately by providing column details.
- Handle headers, data types, encodings properly.
- Inspect data summary after import to validate.
- Split data into train and test sets for modeling.
- Follow best practices for clean and smooth data import.
Once data is activated, you can use the powerful H2O algorithms like GBM, RF, and Deep Learning to train machine learning models on your data at scale. The broad range of data import options make it easy to get started with H2O for AI applications.
