What is Qlik Deployment Framework (QDF)?
Qlik Deployment Framework (QDF) is a collection of methodologies, best practices, and tools designed to facilitate the efficient deployment and management of Qlik Sense implementations for business intelligence purposes. QDF encompasses various aspects of Qlik deployment, including architecture design, application development, security implementation, and deployment automation. It offers a structured approach to Qlik deployment, enabling organizations to optimize performance, scalability, and governance while reducing deployment time and effort.
Key Components of Qlik Deployment Framework:
- Architecture Design: QDF offers guidelines for scalable and resilient Qlik Sense architectures, accommodating diverse data volumes and user loads. It emphasizes data architecture, including data modeling best practices and optimized structures, aligning deployments with broader data strategies.
- Application Development: QDF offers recommendations for building Qlik Sense applications that adhere to industry best practices. It covers data modeling, scripting, visualization design, and application optimization techniques, enabling developers to create high-quality applications that deliver actionable insights.
- Security Implementation: QDF includes guidelines for implementing robust security measures within Qlik Sense applications. These recommendations cover role-based access control, data encryption, authentication mechanisms, and data governance practices. By adhering to standardized security protocols, organizations can ensure data privacy and compliance with regulatory requirements.
- Deployment Automation: QDF offers tools and utilities to automate the deployment process of Qlik Sense applications. It supports continuous integration and continuous deployment (CI/CD) pipelines, version control integration, and environment management, streamlining the release and distribution of Qlik applications across different environments.
- Governance and Best Practices: QDF promotes governance best practices to maintain consistency and integrity across Qlik deployments. It includes recommendations for metadata management, data lineage tracking, change management, and documentation, enabling organizations to maintain a well-governed Qlik environment.
Benefits of Qlik Deployment Framework:
- Efficiency: QDF streamlines the deployment process, reducing manual effort and minimizing the risk of errors. By automating repetitive tasks and standardizing deployment procedures, organizations can deploy Qlik applications more efficiently and consistently.
- Scalability: QDF provides guidelines for designing scalable Qlik architectures that can accommodate growing data volumes and user loads. It enables organizations to scale their Qlik deployments seamlessly as their business requirements evolve.
- Governance: QDF promotes governance best practices to ensure data integrity, security, and compliance within Qlik environments. By implementing robust security measures and adhering to governance guidelines, organizations can mitigate risks and maintain trust in their data analytics processes.
- Collaboration: QDF fosters collaboration among development teams, administrators, and business users by providing a common framework for Qlik deployment. It promotes communication, knowledge sharing, and alignment across different stakeholders, enabling organizations to achieve their analytics objectives more effectively.
Functions in QDF:
1. FileExist
This sub function validates if a file or folder exists, and can be used before load to avoid errors during script load.
The FileExist sub-function is designed to validate whether a specified file or folder exists. It's commonly used before loading operations in QlikView or Qlik Sense scripts to prevent errors that might occur if attempting to load data from non-existent files or folders.
Call vL.FileExist('Folder or file to validate'); Returns variable vL.FileExist with true or false
2. LoadVariableCSV
LoadVariableCSV is a sub-routine used in Qlik scripts to load variables stored in CSV files into the Qlik application's data model.
Call LoadVariableCSV('[My Variable File.csv]', [‘Search Tag’], [‘Container Prefix’], [‘Comments as variables’] [‘Container Map Mode’]);
3. LoadContainerGlobalVariables
LCGV (LoadContainerGlobalVariables) is a function in Qlik scripting used to mount global variables onto other containers based on the Container Map, facilitating easy management and access to global variables within Qlik applications.
Call LCGV/LoadContainerGlobalVariables (‘Container Prefix’, [' Specific folder; Additional folders separated by ;],[Alias])
4. DoDir
DoDir is a Qlik scripting function that indexes specified folder structures, returning a table of file names and paths within the selected file system. It simplifies the process of accessing and analyzing file system information in Qlik applications.
Call DoDir (Scan Path, [Table Name], [Folders Only], [Single Folder], [Qualified Felds], [Hide QDF Templates])
5. CreateFolder
The CreateFolder function in Qlik scripting creates a folder or folder structure if it doesn't already exist, simplifying the process of managing directory creation within Qlik applications.
sub CreateFolder (vL.FolderName)
6. CalendarGen
CalendarGen is a master calendar function initially developed by Jonas Valleskog and further enhanced by Qlik, facilitating the creation and management of generic calendars within Qlik applications. It enables scalable handling of multiple date fields and offers native support in Qlik Sense, providing dynamic and selectable calendar functionality within the application interface.
CALL CalendarGen('Date Field',[ 'Calendar Table'] [,‘Months Left Fiscal Date’] [,‘Min Date’, ‘Max Date’][,’Link Table’][,’DateFormat’];
7. QVFileInfo
The QVFileInfo sub-function is a utility in Qlik scripting that retrieves metadata information in table format about Qlik files, specifically QVW (QlikView) and QVD file formats. It provides details about the files, such as their size, creation date, and other relevant attributes. Notably, it does not support QVF (Qlik Sense) files. This functionality aids in understanding and managing Qlik assets within the application environment.
Call QVFileInfo( 'Fully Qualified file Name',['Table Name'])
8. QVDMigration
QVDMigration is a sub-function in Qlik scripting that facilitates the migration and consolidation of QVD (QlikView Data) files between containers. It supports migrating individual files using fixed file names or entire folders using wildcard (*) notation. Additionally, QVDMigration offers options to migrate selected fields and scramble fields if required. This function is primarily designed for data migration into a self-service (sandbox) environment, with automatic creation of needed subfolders in the destination path facilitated by the CreateFolders function.
Call QVDMigration (QVD Source File, QVD Destination File, [Select specific fields (, separator) leave blank for all fields], [Scrambled fields (, separator)], [Table Name Suffix], [Include Subfolders], [Format-Spec], [No of Records]);
9. QVDLoad
QVDLoad is a function in Qlik scripting that loads QVD (QlikView Data) files into a data model based on the metadata headers within the QVD files. It utilizes the same code and switches as QVDMigration, allowing optional loading of QVD files stored in subfolders, with the only difference being the destination path. This function streamlines the process of loading QVD data into Qlik applications while offering flexibility for organizing and managing data stored in QVD files.
Call QVDLoad(QVD Repository, [Select specific fields (, separator) leave blank for all fields], [Scrambled fields (, separator)], [Table Name Suffix], [Include Subfolders], [No of Records]);
10. DynamicContainerGlobalVariables
DCGV (DynamicContainerGlobalVariables) is a function in Qlik scripting that facilitates the creation of global variables by "mounting" folders and establishing GlobalVariable links. Unlike LCGV, which relies on a container folder mapping table, DCGV generates global variables based on folder names within other containers. Additionally, DCGV allows for the creation of global variable mounts to optional folder structures, extending its utility beyond containers. This function is particularly useful when transitioning to a Qlik Deployment Framework (QDF) environment, enabling the masking of the old environment as a container within the new framework.
Call DCGV(‘Container Name or URL’, [' Specific Folder [; Additional folders separated by ;]'] , [’Override Prefix’]);
11. IndexAdd
IndexAdd is a function in Qlik scripting that generates a QVD index based on metadata and tags extracted from the QVD header. This index serves as a reference point for the QVD file, facilitating efficient data retrieval and manipulation. By utilizing metadata and tags, the index can be recreated if necessary, providing resilience to data structure changes. By default, the index is stored in the $(vG.SharedConfigPath)/Index directory, although this location can be customized as needed to align with organizational requirements. Overall, IndexAdd enhances data management capabilities by optimizing data access and providing a mechanism for maintaining data integrity within Qlik applications.
Call IndexAdd([‘QVD path and name’],['Index folder name'] ,['Container name'] ,['Tags'] ,['Alternative Index path'])
12. IndexLoad
IndexLoad is a function in Qlik scripting used to load QVD (QlikView Data) data based on specified index search criteria, such as tags and field names. The primary search criteria for IndexLoad is the QVD field name, thus it's advisable to adhere to a QVD naming convention, such as day-month-year-TableName.qvd. This function simplifies the process of retrieving data from QVD files by utilizing metadata and field names, enhancing data access and manipulation within Qlik applications.
Call IndexLoad([‘QVD file name’],['Table Name'] ,['Index Folder Name'] ,['Tags'] ,['Fields'] ,['Index
Only'] ,['Load Max Rows'] ,[ ['Alternative Index path'],[‘Load Expressions’])
13. IndexDel
IndexDel is a function in Qlik scripting used to delete indexes and optionally associated QVD (QlikView Data) files. It is particularly useful for maintaining a consistent QVD strategy, such as managing historical data retention. For instance, if storing 24 months of historical indexes and QVD files, IndexDel can be used to remove files older than 24 months to prevent unlimited growth of historical data. IndexDel searches for the QVD field name from the index, emphasizing the importance of adhering to a QVD file naming convention, such as TableName\day-month-year-TableName.qvd. This function helps ensure efficient data management and control within Qlik applications.
Call IndexDel(‘Index file name’,['Delete associated QVD files'] ,['Index Folder Name'] ,['Alternative Index path'])
14. NiceTrace
NiceTrace is a sub-function in Qlik scripting designed to enhance the appearance and readability of text output in the reload progress window and logs of Qlik. It enables the generation of timestamped trace messages with customizable prefixes, making them easier to identify and interpret during the reload process. NiceTrace offers a convenient way to provide informative and visually appealing trace messages, enhancing the monitoring and debugging capabilities within Qlik applications.
call vL.NiceTrace('message to show on console and in the logs')
15. reduceQVD
ReduceQVD is a sub-routine in Qlik scripting designed to reduce the size of temporary QVD (QlikView Data) files generated as part of the ETL (Extract, Transform, Load) process. This function addresses the challenge in Qlik Sense where there is no native support for deleting files from the operating system. By reducing the size of temporary QVD files, ReduceQVD helps prevent excessive consumption of disk space, ensuring efficient storage management within the Qlik environment.
Call reduceQVD ('Path to QVD file','QVD filename')
16. LoadContainerMap
LoadContainerMap is a function in Qlik scripting utilized to load the Container Map CSV file and retrieve information specific to a designated container. This function primarily serves as an internal tool for other functions, such as LCGV, to establish Global Variable links within the Qlik environment. By loading and parsing the Container Map CSV file, LoadContainerMap facilitates the creation of organized and structured connections between containers and their associated resources, enhancing the efficiency and management of Qlik applications.
sub LoadContainerMap(‘Container Map file’, ’Container name’, [' Optional $(vG.BasePath)']);
Conclusion
To summarize, Qlik Deployment Framework (QDF) streamlines Qlik Sense implementations by providing standardized guidelines across architecture, development, security, and automation.
In order to maximize value, it aligns deployments with more comprehensive data strategy and optimizes performance, scalability, and governance.
