Migrating Custom Search Definition and Configuration

In the following DIY blog series, I described how we can use the framework provided in PeopleTools 8.58 to configure our own custom visualizations and dashboards in Kibana and use them in PeopleSoft applications via the 'Kibana Visualizer'.

Refer: DIY - Kibana Dashboards with Application Data

Someone ask a very valid question:
How do we migrate these configurations from one environment (source) to another (target)?

This post will detail the steps for migrating the Search Definition and associated configuration.

Migration

We can migrate everything related to the Search Definition using an App Designer project. Simply insert the Search Definition and make sure the the 'Related Definitions' are selected as shown below.

Save the 'Project' and migrate to target environment.

Note: If we used any Application Package PeopleCode to configure the Search Definition (Security Configuration) - Document Filter App Class, then we must also include the App Package in the project.

Post Migration Steps

Once we migrate the configuration, we must complete the following administrative steps in the target environment.

1. Deploy Search Definition
2. Schedule Search Index
3. Configure Search Context (optional)
4. Configure Search Group Security

Related Posts

Migrating Custom Kibana Visualizations/Dashboards

Access Kibana (Admin) Home from PeopleSoft

In the following DIY blog series, I described how we can use the framework provided in PeopleTools 8.58 to configure our own custom visualizations and dashboards in Kibana and use them in PeopleSoft applications via the 'Kibana Visualizer'.

DIY Kibana Dashboards with Application Data

Kibana Admin Login

As part of the DIY configuration process (detailed in Part 2), we need to login to the Kibana Administration page and setup index patterns, visualizations and dashboards. Typically, this step would/should be completed by a developer/analyst/admin type of user. As you can see below, the Kibana Administration page requires a separate login. With the latest integration provided in PeopleTools, we can simply use the same userid/password as in PeopleSoft and also enter the database name.

Access Kibana Administration from PeopleSoft using SSO

PeopleSoft already delivered a security plug-in for Kibana that allows end users to access Kibana visualizations and dashboards from within the PeopleSoft UI. Basically, if users are logged in to PeopleSoft (with a PS_TOKEN), then they can simply access the Kibana visualizations without having to login to Kibana again.

We can take the same logic and create a link in PeopleSoft that will open the Kibana (Admin) Home without requiring a separate step to login to Kibana. In the screenshot below, we can see the user and token parameters in the Kibana iframe URL.

The logic for generating this URL is available in PTSF_KIBANA_COMP.Activate Page PeopleCode.

Custom IScript

We can take the same logic available in the delivered Page PeopleCode and create a custom IScript to redirect to the Kibana (Admin) Home.

Custom Content Reference (CREF)

Next, we can create a custom content reference and associated the IScript created in the previous section. For the sake of simplicity, I added the CREF under PeopleTools > Search Framework > Administration folder. You can change this to an appropriate parent folder of your choice.

Demo

DIY - Kibana Dashboards with Application Data - Part 3

This post is part of a series: DIY - Kibana Dashboards with Application

The major parts of creating a search index (using application data) in Elasticsearch and creating Dashboards (and visualizations) in Kibana were completed in Part 1 and Part 2. This part will focus on the final steps of importing the Kibana Dashboards (and visualizations) into PeopleSoft, configuring and deploying them as PeopleSoft visualizations (tiles, related content, etc.).

Import Kibana Dashboards

In Part 2, we created two dashboards (Full View and Tile View) in Kibana. These dashboards can be imported into PeopleSoft from PeopleTools > Search Framework > Administration > Import Kibana Dashboards.

Configure the Dashboard(s) as a Tile

Once the Dashboards are successfully imported into PeopleSoft, we can configure them as a Tile (or related content) from PeopleTools > Search Framework > Administration > Kibana Visualizations.

Note: When we setup the dashboard as a Tile, PeopleTools will automatically create the Tile content reference for us as shown below.

Specify User Privileges

Once we configure the Kibana dashboard(s) as a Tile, we can specify the user privileges to secure access to the Tile.

PeopleTools > Search Framework > Administration > Kibana Privileges

Note: This will automatically update the Tile content reference security appropriately.

Deploy Kibana Dashboards

Finally, we can deploy (redeploy?) the dashboards configured in PeopleSoft as a visualization (Tile) into Kibana. This is step will deploy all the configurations (security, etc.) associated with the dashboards in Kibana.

PeopleTools > Search Framework > Administration > Deploy Kibana Dashboards

DIY - Kibana Dashboards with Application Data - Part 2

This post is part of a series: DIY - Kibana Dashboards with Application

Once the search index (based on application data) is deployed and functional in Elasticsearch, we can now move ahead with the process of configuring visualizations and dashboards in Kibana.

Create Index Pattern in Kibana

At this stage, we know that our custom search index is working in Elasticsearch. The next step is to create an index pattern in Kibana. This index pattern will then become the hook (connection) between the search index in Elasticsearch and the visualizations in Kibana.

Let us login to Kibana (using PS userid) and create the index pattern based on the search index in Elasticsearch.

Accessing Kibana Dashboards from PeopleSoft

PS userid has elevated security in the application so we know that we can login to Kibana without any issues using that userid. But what is the security required to be able to login to Kibana?

PeopleBooks: Accessing Kibana Dashboards from PeopleSoft

"A PeopleSoft user with the Search Administrator role or a user who has edit privilege or create privilege to any of the dashboard can log on to Kibana directly. The edit or create privilege is specified on the Kibana Privileges page."

After logging into Kibana as PS userid (or other userid with appropriate access), we can create the index pattern based on the search index as shown below.

Kibana > Management > Index Patterns > Create Index Pattern

Create Visualizations

Now that we have everything in place, we can start creating our visualizations in Kibana.

Tag Cloud Visualization

Horizontal Bar Visualization

Vertical Bar Visualization

Pie Visualization

Saved Search with Drilling URL

Before we create a saved search with drilling URL, we must change the format of the drilling URL field from string to URL. This will ensure that the URL is actually converted into a hyperlink in the visualization.

Kibana > Management > Index Patterns > Select Index Pattern: sv_ib_async_msg* > Field: ORCL_ES_LINK

Now we can go ahead and create a saved search.

Create Dashboards

The first dashboard created in the video shown below will be the detailed dashboard (Full View) which contains all the visualizations.

The next video demonstrates how to create a Dashboard with a single visualization. This will be used as the Tile Dynamic Content for the Kibana Visualizer (Tile View).

That is it! Our job within Kibana is completed.

DIY - Kibana Dashboards with Application Data - Part 1

This post is part of a series: DIY - Kibana Dashboards with Application

Create Search Definition

Create Search Query

PeopleBooks: Understanding Search Definition Queries

We will expose the required application data (PSAPMSGPUBHDR table in this example) as a search index by creating a query and referencing the query in the search definition.

From PeopleBooks: "The data returned by the query/connected query serves as the source data that the search engine crawls to create the search index."

The requirements to build a query (based on application data) are simple and shown below.

Create Search Definition

PeopleBooks: Creating Search Definitions

A simple search definition was created using the query created earlier. For the purpose of this post, we will not set any security on this search definition. Basically, the search definition will be accessible as long as we have access to the application.

Create Search Category

PeopleBooks: Understanding Search Categories

A simple search category was created referencing the search definition from the previous section. The 'Search Group' checkbox in the 'General' tab allows us to enable this search index via Global Search.

Deploy Search Definition

PeopleBooks: Administering Search Definitions and Search Categories

From PeopleBooks: "Before end users can run searches against the search indexes, the search definitions need to be deployed to the search engine so that the search engine can create the structure of the search index based on the search definition, crawl the defined search criteria, and populate the index with the results of the search definition query."

We simply need to navigate to PeopleTools > Search Framework > Administration > Deploy/Delete Object, select our search definition and 'Deploy'. The status of our search definition will change to 'Deployed' and 'Not Crawled'.

Schedule Search Index

Next we can schedule the search index build process from PeopleTools > Search Framework > Administration > Schedule Search Index.

Once PTSF_GENFEED process completes successfully, we should see the status of our search definition deployment change to 'Deployed' and 'Successful'.

PeopleTools > Search Framework > Administration > Deploy/Delete Object

Define Search Context

Our Search Category must be added to the 'Homepage' Search Context to enable our search to appear as part of Global Search.

PeopleTools > Search Framework > Administration > Define Search Context

Setup Search Group Security

Finally, we need to also ensure that the Search Group is added to an appropriate Permission List to enable access via Global Search.

Refer: E-ES/SES: Search Category not Showing Up in Global Search Bar (Doc ID 1534166.1)

Verify Search Index

We can verify that our search index is functional either using the 'Search Test Page' or using the Global Search as shown below.

DIY - Kibana Dashboards with Application Data

In my recent post, I wrote about getting started with Kibana dashboards in PeopleTools 8.58 using application data. In that post, we saw how we can deploy and use some of the delivered Kibana dashboards that were built using application data (search indices).

Refer: PeopleTools 8.58 - Application Data and Kibana Dashboards

While it is nice to use some of the delivered dashboards as examples to kick the tires, we don't have to wait for Oracle/PeopleSoft to deliver search indices (based on application data) and Kibana dashboards (based on those search indices) to take advantage of this functionality. The entire framework is available in PeopleTools! Why not build (configure) our own from scratch?

This post will provide details on how we can do just that.

Demonstration

The below video shows a completely custom Kibana Dashboard configured as a Tile and rendered using the Kibana Visualizer delivered in PeopleSoft.

In this example, I used Integration Broker asynchronous message data (PSAPMSGPUBHDR table) as the 'application data' use case.

Integration Broker > Service Operations Monitor > Monitoring > Asynchronous Services

Steps to Do It Yourself

PeopleBooks: Understanding Application Data and Kibana Dashboards

The PeopleBooks referenced above describes the process of creating visualizations in Kibana and making them available as tiles (or related content). Before we begin with the first step of creating visualizations and dashboards in Kibana, we must create and deploy a search definition based on the required application data, crawl the search index and ensure that it is available in Elasticsearch. This is required because the Kibana visualization will be built based on this application search index.The below posts detail the end to end process in three parts.

DIY - Kibana Dashboards with Application Data - Part 1

This post covers how we can create a search definition based on application data, deploy the search index on Elasticsearch and verify the search index functionality.

DIY - Kibana Dashboards with Application Data - Part 2

This post covers how we can create Visualizations and Dashboards in Kibana using PeopleSoft application data (search indices).

DIY - Kibana Dashboards with Application Data - Part 3

This post will cover how we can import Kibana Dashboards (and Visualizations) into PeopleSoft, configure and deploy them as PeopleSoft visualizations (tiles, related content, etc.).

Updated on June 12th, 2020

Here is a video of my presentation - DIY Kibana Dashboards at Education Innovation Sessions hosted by CY2 and partners.

https://www.youtube.com/watch?v=awiL3f7j4KI

PSPRSMNAVINFO - Portal Menu Navigation in SQL

Last week, I worked on a requirement where I needed to find all the components that are part of a role (or a list of roles) including the Portal Menu Navigation Path if it exists. It is easy to get the list of components that are part of a role but it is not really straight forward to qualify that information with the actual Portal Menu Navigation (if it exists).

 In the past, I used the following query to return the Portal Menu Navigation Path for a Component. The query mines through the Portal Structure and Content and presents the navigation hierarchy in the result.

As we can see, it will not be easy to re-purpose this hierarchical query and expand it for my requirement.

While poking around on this topic, I stumbled on this gem of a table PSPRSMNAVINFO that, I believe, was introduced thanks to the PeopleSoft Search Framework!

Here is what we can see in the Definition Properties of this record:
"Holds the Navigation Path to the given target crefs registered in Portal Registry. Holds only effective and non-expired target crefs.
Used by the Portal Menu SES Search.
Updated every schedule of search index file"

From my understanding, it appears that this table is used as part of the PeopleSoft Search Framework - PTPORTALREGISTRY Search Definition. Also, the App Engine library PORTALPATHAE is responsible for populating this table with the navigation path information when the search index is built.

PeopleTools > Search Framework > Designer > Search Definition (PTPORTALREGISTRY)

PeopleTools > Search Framework > Administration > Schedule Search Index

To identify the last time the PTPORTALREGISTRY search definition index was built, we can query the table PSPRSMPATHSTAT.

If for any reason we don't have the Search Framework enabled and we still want to take advantage of the data in this table, then we can simply populate this table by directly running the PORTALPATHAE App Engine Library from App Designer. Since I was testing this on a Virtual Box installation of a PUM image (CS 9.2 - Image 4), I did not have Search enabled/configured. I ran the AE from App Designer as follows:

Now, let us see how we can use PSPRSMNAVINFO to get the Portal Menu Navigation information.

The best part about this table is that it allows us to directly retrieve the navigation path for a component without having to deal with the hierarchical structure in PSPRSMDEFN. This makes it easier to use/expand this information in other queries.

Here is how I extended this information to create a query that retrieves all components in a role along with the Portal Menu Navigation Path information (if it exists).


Github link to SQL Queries in this post:
https://github.com/SasankVemana/PS-Portal-Menu-Navigation-Queries