feat: MTP support

[JoshuaWise]

Feature Description

node-llama-cpp supports its own hand-rolled version of speculative token prediction via DraftSequenceTokenPredictor, but llama.cpp now supports this natively, without requiring a second model GGUF to be loaded.

The Solution

Add bindings for using the native MTP provided by llama.cpp: ggml-org/llama.cpp#22673

Considered Alternatives

I considered DraftSequenceTokenPredictor, but the native implementation is expected to perform better, and has a more convenient API (single GGUF) compared to DraftSequenceTokenPredictor.

Are you willing to resolve this issue by submitting a Pull Request?

No, I don’t have the time and I’m okay to wait for the community / maintainers to resolve this issue.

[giladgd]

I recently answered a similar question regarding MTP support in node-llama-cpp: #625 (reply in thread)

I plan to add support for this in v4 (that I'm working on for a while now) as it requires some architectural changes to work well.
Also, many models ship the MTP model as a separate gguf, so the model downloader will have to be rearchitected as well to support this and a few other incoming features.

[JoshuaWise]

Okay, thanks for the update!