•   about 2 years ago

Improve Gemini processing speed

Hello,

I'm using 'gemini-pro-vision' to process an image, but it takes 10 - 15 seconds for Gemini to process it. My prompt is just 1 sentence long and that's not too much. Is there any way I could improve the performance of Gemini? Are there any documentation on this?

I'm using these configs:

@staticmethod
def create_generation_config():
return GenerationConfig(
temperature=0.4,
top_p=1,
top_k=32,
candidate_count=1,
max_output_tokens=4096,
)

@staticmethod
def create_safety_config():
return {
HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_NONE,
HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_NONE,
HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_NONE,
HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_NONE,
}

Thanks

  • 0 comments

Comments are closed.