Thursday, May 21, 2020

DIY - Kibana Dashboards with Application Data - Part 1

This post is part of a series: DIY - Kibana Dashboards with Application

Create Search Definition

Create Search Query

PeopleBooks: Understanding Search Definition Queries

We will expose the required application data (PSAPMSGPUBHDR table in this example) as a search index by creating a query and referencing the query in the search definition.

From PeopleBooks: "The data returned by the query/connected query serves as the source data that the search engine crawls to create the search index."

The requirements to build a query (based on application data) are simple and shown below.


Create Search Definition

PeopleBooks: Creating Search Definitions

A simple search definition was created using the query created earlier. For the purpose of this post, we will not set any security on this search definition. Basically, the search definition will be accessible as long as we have access to the application.


Create Search Category

PeopleBooks: Understanding Search Categories

A simple search category was created referencing the search definition from the previous section. The 'Search Group' checkbox in the 'General' tab allows us to enable this search index via Global Search.


Deploy Search Definition

PeopleBooks: Administering Search Definitions and Search Categories

From PeopleBooks: "Before end users can run searches against the search indexes, the search definitions need to be deployed to the search engine so that the search engine can create the structure of the search index based on the search definition, crawl the defined search criteria, and populate the index with the results of the search definition query."

We simply need to navigate to PeopleTools > Search Framework > Administration > Deploy/Delete Object, select our search definition and 'Deploy'. The status of our search definition will change to 'Deployed' and 'Not Crawled'.


Schedule Search Index

Next we can schedule the search index build process from PeopleTools > Search Framework > Administration > Schedule Search Index.


Once PTSF_GENFEED process completes successfully, we should see the status of our search definition deployment change to 'Deployed' and 'Successful'.

PeopleTools > Search Framework > Administration > Deploy/Delete Object


Define Search Context

Our Search Category must be added to the 'Homepage' Search Context to enable our search to appear as part of Global Search.

PeopleTools > Search Framework > Administration > Define Search Context


Setup Search Group Security

Finally, we need to also ensure that the Search Group is added to an appropriate Permission List to enable access via Global Search.


Refer: E-ES/SES: Search Category not Showing Up in Global Search Bar (Doc ID 1534166.1)

Verify Search Index

We can verify that our search index is functional either using the 'Search Test Page' or using the Global Search as shown below.


14 comments:

  1. Hi Sasank,

    I am just getting one row in Global Search. I did everything same from your DIY. Could you please also share the PS Query created.

    Thanks
    Payal

    ReplyDelete
    Replies
    1. Hi Payal - You might want to compare how many rows of data you are seeing in Integration Broker (directly by navigating to the page). This way, you can compare the rows that are in Elasticsearch.

      Also, the amount of data you see in Elasticsearch depends on the date range you used for the Search Indexing.

      Delete
    2. I was doing mistake with Drilling URL. Thanks it worked out well now.

      Delete
    3. Great! Thanks for letting us know.

      Delete
    4. @Payal - Out of curiosity, what was the issue with the Drilling URL? How did you resolve it?

      Delete
    5. Sorry for late reply.

      Actually, Drilling URL was not showing me URL Keys so when I clicked the URL it was taking me to search page.
      Then, I used fields from "Select Field" and gave them order to use as URL Keys. It worked.

      Delete
  2. Hi Sasank,

    I am trying to create a CFO dashboard in Kibana which requires combining items across multiple indexes on Kibana. For example, Quick Ratio = (Current Assets – Inventories – Prepaids) / Current Liabilities. These fields are scattered across AR, AP, GL, etc indexes. How can I combine them together to vizualize this data on Kibana?

    Really appreciate your help.

    Regards,
    Pramod

    ReplyDelete
    Replies
    1. Hi Pramod - You will need to build a connected query with all the data that you require. You may choose to build a single index or create multiple indexes per module. Then it is a matter of creating an index pattern either per index or pattern matching all the indexes.

      I have not tried this but it should work in theory.

      Delete
    2. Thank you so much for getting back to me. I have tried several ways to achieve this but I am constantly hitting roadblocks. The Elastic search documentation is vast but not very clear at times. It would be really helpful if you could show us a demonstration. Thank you in advance!

      Regards,
      Pramod

      Delete
  3. Hi Sasank,

    I am creating a Kibana viz for Timesheet Source.
    I have a requirement wherein a consultant X should only have access to Timesource data of employees working under it. X should only see visualizations of its subordinates. As of now, on Kibana Dashboard X is able to see all the data indexed.

    But, if I try handling the security from PS Query (Data Source for Kibana), it will not index all the data.

    Please let me know, if there’s any way to achieve this.

    Thanks,
    Payal

    ReplyDelete
  4. Hi Sasank,

    Just wondering if I want to have index on data that doesnt have datetime field then how should we do that?
    I understand that we wont be able to get incremental index.

    ReplyDelete
    Replies
    1. Hi Rohil - Good question. It is not possible to create an index without a datetime stamp field. It is requirement of Elasticsearch.

      That said, I have seen some delivered search definitions using views to derive the datetime stamp from other tables/logic.

      Hope this gives you some ideas.

      Delete
  5. Hi Sasank, Thank you for such a wonderful post. I am trying to create an index based on a query as data source, but PTSF_GENFEED is only indexing a fraction of data (few hundred rows against thousands of rows from the output of the query) even when its run FULL INDEX mode.

    What can be the probable reason? how to debug this?

    Thank You,

    ReplyDelete
    Replies
    1. Please see this thread on MOSC:
      https://community.oracle.com/mosc/discussion/comment/16893992

      Delete