[PoC ONLY] Qualcomm AI Engine Direct - HF Exporter Mininal Script

[winskuo-quic]

Summary

This PR serves as a PoC demo how to use hf-transformer exporter to export a HF LLM model.
Since this is just a PoC, a lot of stuff in the patch and this PR is hardcoded. e.g., the example script is set to Llama3.2 1B. A lot of implementations for hf-transformer exporter patch is hardcoded and not the best design, it is just to prove we can get the same performance using hf-transformer exporter flow.

Steps

Notes: All these will be done in a single conda environment, you don't need to create multiple environments.

Step 1

Checkout to this branch. Enter your conda environment, do all the basic setups as usual, e.g., PYTHONPATH=$EXECUTORCH_ROOT/.., git submodule sync, git submodule update --init, python install_executorch.py, etc.

Step 2

Next, create an empty folder and inside do the following.
git clone the hugging face repo to your local machine: https://github.com/huggingface/transformers

Step 2

Checkout to this PR: huggingface/transformers#47747

Step 3

Apply the PoC patch to HF repo, under ./hf-transformers/transformers

Step 4

After you apply the patch, do:
pip uninstall -y transformers # Probably optional buy highly recommended
pip install -e . --no-deps # --no-deps in case it overrides some executorch dependencies.

Step 5

Execute
python examples/qualcomm/oss_scripts/hf_exporter.py --prompt "Simply put, the theory of relativity states that" --soc_model SM8750 --device $DEVICE --build_folder build-android/ --max_seq_len 1024

Performance Check

This is a quick to ensure that the model lowered through HF-Exporter API is similar as ExecuTorch directly lowering HF model, we have checked the final performance and compared the number of Executorch nodes. After checking both, the node number and performance does match.

Original Flow

HF-Exporter Flow

However, this is only for quick validation and does not guarantee the final QNN graph is identical.

Test plan

NA

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22400

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures

As of commit fe92052 with merge base d750618 ():

NEW FAILURES - The following jobs have failed:

• pull / test-qnn-passes-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 38faf261ad0b2c86f8849abf6ce309162a55a6a7bc9c44687bb31ce9d6b16d05 /exec failed with exit code 4
• pull / unittest / linux / linux-job (gh)
  pytorch_tokenizers
• pull / unittest-buck / linux / linux-job (gh)
  pytorch_tokenizers
• pull / unittest-editable / linux / linux-job (gh)
  pytorch_tokenizers

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[winskuo-quic]

Hi @psiddh,
This is a PoC to demo how to run HuggingFace Exporter API with ExecuTorch. Please take a look at the summary section of this PR to see how to reproduce.
It requires applying another patch to HuggingFace repo.

Overall, we followed HuggingFace convention and design, using only this single API call to lower a model to backend.

This PR looks a little messy because I applied this on top of my HF model optimization PR: #22170, so please ignore the 2nd commit and just focus on the top commit of this PR.
Thanks