Release 0.4.0 (Oct. 4, 2016)¶
Highlights¶
- Support Streaming/Rapid Fire feeds from NiFi
Note
Operations Manager User Interfaces for viewing Streaming feeds will come in a near future release.
- Security enhancements including integration with LDAP and administration of users and groups through the web UI
- Business metadata fields can be added to categories and feeds
- Category and feed metadata can be indexed with Elasticsearch, Lucene, or Solr for easier searching
- Fixed bug with kylo init.d service scripts not support the startup command
- Fixed issues preventing preconditions or cleanup feeds from triggering
- Fixed usability issues with the visual query
- Better error notification and bug fixes when importing templates
- Service level agreement assessments are now stored in our relational metadata store
- Spark Validator and Profiler Nifi processors can now handle additional Spark arguments
- Redesign of job details page in operations manager to view steps/details in vertical layout
- Allow injection of properties for any processor or controller service in the application.properties file. The feed properties will be overridden when importing a template. This includes support to auto fill all kerberos properties.
Known Issues¶
- The Data Ingest and Data Transformation templates may fail to import on a new install. You will need to manually start the SpringContextLoaderService and the Kylo Cleanup Service in NiFi, then re-import the template in the Feed Manager.
- When deleting a Data Transformation feed, a few Hive tables are not deleted as part of the cleanup flow and must be deleted manually.
Running in the IDE¶
If you are running things via your IDE (Eclipse or IntelliJ) you will need to run the following command under the core/operational-metadata/operational-metadata-jpa module
mvn generate-sources
This is because it is now using JPA along with QueryDSL(http://www.querydsl.com/), which generates helper Query classes for the JPA entities. Once this runs you will notice it generates a series of Java classes prefixed with “Q” (i.e. QNifiJobExecution) in the core/operational-metadata/operational-metadata-jpa/target/generated-sources/java/
Optionally you could just run a mvn install on this module which will also trigger the generate-sources.
Additionally, if you havent done so, you need to ensure the latest nifi-provenance-repo.nar file is in the /opt/nifi/data/lib folder.