Awesome

OpenChatLog: A Search Engine for LLM Generated Texts

Overview

We build a new search engine on large language model (LLM) generated texts using data from the ChatLog dataset.

There are 2 query types for users to customize:

• Given a question, search answer candidates from our LLM histroy responses database by question matching.
• Given a piece of text (expected answer from LLM), search the most matching LLM responses in database.

What's the difference?

Compare our search engine with LLM, there are some advantages on the deployment:

Also, there are some advantages and disadvantages on the service:

Start

Start the server:

streamlit run OpenChatLog.py --server.port 9621

Visit the website at

http://103.238.162.37:9621

Collect Data

Data Sources

1.Collect from open-source datasets:

2.Collect by calling the api:

gpt-3.5-turbo

The data will be continuously updated by Monthly and Daily.

Data format

Save data by:

python -m data.save_data --source_dataset HC3

Build Backend

Use elasticsearch for our backend server.

Start it at /data/tsq/elasticsearch-7.15.2/bin:

./elasticsearch -d

View logs at /data/tsq/elastic_data/logs

The bind port is 9344.

Then, we need to index all the qa pairs using the elasticsearch.

nohup python -m data.backend --source_dataset Jack_of_all_trades &

• GPTeacher
• medAlpaca
• UltraChat
• ArguGPT
• GPT-4-LLM
• mwp
• robustlearn
• Jack_of_all_trades
• HC3

Build Front Page

Use text_input to get the search query.

Use selectbox to get the query type.

Finally, we can open our server by:

nohup streamlit run OpenChatLog.py --server.port 9621 > server.log &

Evaluation

Use 3 OpenQA dataset:

Search in OpenChatLog:

python -m eval.main --dataset nq --task step1 --split test --num_sequence 25

Search in answers of OpenChatLog:

python -m eval.main --dataset nq --task step1 --split test --num_sequence 25 --search_field a

compare with:

https://arxiv.org/pdf/2101.00408.pdf

Result are:

Misc

<span style="background-color:rgb(100,200,200,0.5)">Me:</span> Give me some ideas about creating a search engine on ChatGPT's responses corpus

<span style="background-color:rgb(100,200,200,0.5)">Bard:</span> Here are some ideas about creating a search engine on ChatGPT's responses corpus:

1. Index the corpus. The first step is to index the corpus, which means creating a searchable database of all the text in the corpus. This can be done using a variety of techniques, such as full-text indexing or inverted indexing.
2. Create a search interface. Once the corpus is indexed, you need to create a search interface that allows users to search for the information they are looking for. This interface should be easy to use and should allow users to search by keyword, phrase, or concept.
3. Rank the results. Once a user has submitted a search query, the search engine needs to rank the results in order of relevance. This can be done using a variety of techniques, such as term frequency-inverse document frequency (TF-IDF) or PageRank.
4. Provide feedback. The search engine should provide feedback to users about the results of their search. This feedback can be in the form of relevance scores, snippets of text, or links to related documents.

Here are some additional ideas that may be helpful:

• Use a variety of sources. The corpus should be indexed from a variety of sources, such as books, articles, websites, and social media. This will help to ensure that the search engine is comprehensive and up-to-date.
• Use machine learning. Machine learning can be used to improve the accuracy and relevance of the search results. For example, machine learning can be used to identify patterns in the corpus and to rank the results accordingly.
• Make the search engine multilingual. The search engine should be multilingual so that it can be used by people from all over the world. This can be done by indexing the corpus in multiple languages or by using machine translation to translate the results into the user's preferred language.

By following these ideas, you can create a search engine that is comprehensive, accurate, and easy to use.

Future Work

FreeAIGC: Add multi-model AI generated contents for search

Citation

Please cite our paper if you use OpenChatLog in your work

@article{tu2023chatlog,
  title={ChatLog: Recording and Analyzing ChatGPT Across Time},
  author={Tu, Shangqing and Li, Chunyang and Yu, Jifan and Wang, Xiaozhi and Hou, Lei and Li, Juanzi},
  journal={arXiv preprint arXiv:2304.14106},
  year={2023}
}