LLMLingua：集成LlamaIndex，对提示进行压缩，提供大语言模型的高效推理

# Install dependency.
 !pip install llmlingua llama-index openai tiktoken -q 
 
 # Using the OAI
 import openai
 openai.api_key = "<insert_openai_key>"

!wget "https://www.dropbox.com/s/f6bmb19xdg0xedm/paul_graham_essay.txt?dl=1" -O paul_graham_essay.txt

from llama_index import (
    VectorStoreIndex,
    SimpleDirectoryReader,
    load_index_from_storage,
    StorageContext,
 )
 
 # load documents
 documents = SimpleDirectoryReader(
    input_files=["paul_graham_essay.txt"]
 ).load_data()

index = VectorStoreIndex.from_documents(documents)
 
 retriever = index.as_retriever(similarity_top_k=10)
 
 question = "Where did the author go for art school?"
 
 # Ground-truth Answer
 answer = "RISD"
 
 contexts = retriever.retrieve(question)
 
 contexts = retriever.retrieve(question)
 
 context_list = [n.get_content() for n in contexts]
 len(context_list)
 
 #Output 
 #10

# The response from original prompt
 from llama_index.llms import OpenAI
 
 llm = OpenAI(model="gpt-3.5-turbo-16k")
 prompt = "\n\n".join(context_list + [question])
 
 response = llm.complete(prompt)
 print(str(response))
 
 #Output
 The author went to the Rhode Island School of Design (RISD) for art school.

from llama_index.query_engine import RetrieverQueryEngine
 from llama_index.response_synthesizers import CompactAndRefine
 from llama_index.indices.postprocessor import LongLLMLinguaPostprocessor
 
 node_postprocessor = LongLLMLinguaPostprocessor(
    instruction_str="Given the context, please answer the final question",
    target_token=300,
    rank_method="longllmlingua",
    additional_compress_kwargs={
        "condition_compare": True,
        "condition_in_question": "after",
        "context_budget": "+100",
        "reorder_context": "sort", # enable document reorder,
        "dynamic_context_compression_ratio": 0.3,
    },
 )

retrieved_nodes = retriever.retrieve(question)
 synthesizer = CompactAndRefine()
 
 from llama_index.indices.query.schema import QueryBundle
 
 
 # postprocess (compress), synthesize
 new_retrieved_nodes = node_postprocessor.postprocess_nodes(
    retrieved_nodes, query_bundle=QueryBundle(query_str=question)
 )
 
 original_contexts = "\n\n".join([n.get_content() for n in retrieved_nodes])
 compressed_contexts = "\n\n".join([n.get_content() for n in new_retrieved_nodes])
 
 original_tokens = node_postprocessor._llm_lingua.get_token_length(original_contexts)
 compressed_tokens = node_postprocessor._llm_lingua.get_token_length(compressed_contexts)

print(compressed_contexts)
 print()
 print("Original Tokens:", original_tokens)
 print("Compressed Tokens:", compressed_tokens)
 print("Comressed Ratio:", f"{original_tokens/(compressed_tokens + 1e-5):.2f}x")

next Rtm's advice hadn' included anything that. I wanted to do something completely different, so I decided I'd paint. I wanted to how good I could get if I focused on it. the day after stopped on YC, I painting. I was rusty and it took a while to get back into shape, but it was at least completely engaging.1]
 
 I wanted to back RISD, was now broke and RISD was very expensive so decided job for a year and return RISD the fall. I got one at Interleaf, which made software for creating documents. You like Microsoft Word? Exactly That was I low end software tends to high. Interleaf still had a few years to live yet. []
 
  the Accademia wasn't, and my money was running out, end year back to the
  lot the color class I tookD, but otherwise I was basically myself to do that for in993 I dropped I aroundidence bit then my friend Par did me a big A rent-partment building New York. Did I want it Itt more my place, and York be where the artists. wanted [For when you that ofs you big painting of this type hanging in the apartment of a hedge fund manager, you know he paid millions of dollars for it. That's not always why artists have a signature style, but it's usually why buyers pay a lot for such work. [6]
 
 Original Tokens: 10719
 Compressed Tokens: 308
 Comressed Ratio: 34.80x

response = synthesizer.synthesize(question, new_retrieved_nodes)
 print(str(response))
 
 #Output
 #The author went to RISD for art school.