Context window 90% of adapter model full after single user prompt

Machine Learning & AI Foundation Models
MichaelOShea OP
Created 2d
Replies 2
Boosts 0
Views 317
Participants 2

I have been able to train an adapter on Google's Colaboratory.

I am able to start a LanguageModelSession and load it with my adapter.

The problem is that after one simple prompt, the context window is 90% full.

If I start the session without the adapter, the same simple prompt consumes only 1% of the context window.

Has anyone encountered this? I asked Claude AI and it seems to think that my training script needs adjusting. Grok on the other hand is (wrongly, I tried) convinced that I just need to tweak some parameters of LanguageModelSession or SystemLanguageModel.

Thanks for any tips.

Replies  2
Boosts  0
Views  317
Participants  2
carinapeng OP
2d

Hey Michael! To better understand your issue, what parameters did you use when training your adapter (max seq length, pack sequence, batch size, etc)? Do you mind copying your AdapterTrainingConfiguration?

Are you using tools in your training data system message? Do you mind sharing a training sample entry?

0
carinapeng OP
8h

Hi! Just want to check if this is still an issue after tool calling succeeded from your other thread post?

0
Context window 90% of adapter model full after single user prompt
First post date Last post date
 
 
Q