Validate Configuration¶
Kylo has many integration points. For example, Hive, NiFi, MySQL, Spark, ActiveMQ, Elasticsearch, etc. Now that we ran a feed through we can test Kylo’s integration with all of these components.
By successfully running Kylo you have validated the MySQL configuration, as well as integration with ActiveMQ and NiFi.
Validate Hive Thrift Connection¶
Test profile statistics:
- Go to the “Feeds” page in feed manager and click on your test feed.
- Go to the “PROFILE” tab and click “view” for one of the rows.
- Go to the “VALID” and “INVALID” tabs and verify data is being returned.
Test the Tables page:
- Go to the “Tables” page in Feed Manager.
- Click on the table for your test feed. In our example it is “test_feed_1”.
- Click the “PREVIEW” and “QUERY” tabs to ensure data is being returned.
Validate Spark Shell¶
- Go to the “Visual Query” page in Feed Manager.
- In the search box type “test_feed_1” then click “add table”.
- Click “Continue to step 2”. Validate you can see data.
- Apply a transformation. An easy way to do this is to click on the “id” column, choose Filter -> “> 600”. Validate you only see numbers greater than 600
Validate Search¶
Validate the schema information is there:
Validate the indexed data is there: