LanguageModelStream and collecting the final output

I have a Generable type with many elements. I am using a stream() to incrementally process the output (Generable.PartiallyGenerated?) content.

At the end, I want to pass the final version (not partially generated) to another function.

I cannot seem to find a good way to convert from a MyGenerable.PartiallyGenerated to a MyGenerable.

Am I missing some functionality in the APIs?

Answered by Frameworks Engineer in 851153022

The ResponseStream.collect() method is designed specifically for this purpose. Once generation is complete, you can call collect() to obtain the final result as the non-partial type.

Accepted Answer

Generable.PartiallyGenerated doesn't have something like asGenerable for now, and that's probably worth a feedback report. (Generable does have asPartiallyGenerated(), btw.)

You are free to write your own asGenerable though. For example:

@Generable
struct Sentiment {
    @Generable
    enum Result: String {
        case positive = "Positive", mixed = "Mixed", negative = "Negative"
    }
    let result: Result
    let reasoning: String
}

extension Sentiment.PartiallyGenerated {
    var asGenerable: Sentiment {
        return Sentiment(result: result ?? .mixed, reasoning: reasoning ?? "")
    }
}

Best,
——
Ziqiao Chen
 Worldwide Developer Relations.

Filed FB19122234!

The ResponseStream.collect() method is designed specifically for this purpose. Once generation is complete, you can call collect() to obtain the final result as the non-partial type.

Indeed. Thanks, @Frameworks Engineer.

Also, @cwoloszynski for awareness – I guess you can close your feedback report if collect(isolation:) fits your use case. Thanks.

Best,
——
Ziqiao Chen
 Worldwide Developer Relations.

LanguageModelStream and collecting the final output
 
 
Q