Skip to content

Improve performance of listTemplates API#13566

Draft
Pearl1594 wants to merge 5 commits into
4.22from
improve-listtemplates-perf
Draft

Improve performance of listTemplates API#13566
Pearl1594 wants to merge 5 commits into
4.22from
improve-listtemplates-perf

Conversation

@Pearl1594

Copy link
Copy Markdown
Contributor

Description

This PR Adds a fast-path for listTemplates Phase 1 that bypasses template_view,
behind a runtime config flag (default off).

Why

The captured incident query against template_view runs 7,000+ seconds and
drains the DB pool. View materializes ~71k rows for ~4.7k templates due to
LEFT-JOIN row multiplication, OR-join on data_center, and predicates on a
computed column. Phase 1 only needs 6 of the 13 tables.

What changed

  • BypassTemplateView ConfigKey (template.list.bypass.view, default false,
    Global, runtime-toggleable).
  • TemplateListFilter POJO + canBypass() predicate.
  • TemplateJoinDao.findDistinctTempZonePairs(filter) — hand-tuned SQL over
    vm_template + account + template_store_ref + image_store + template_zone_ref
    • data_center; OR-join replaced with COALESCE.
  • QueryManagerImpl builds the filter and dispatches to bypass when flag is
    on AND canBypass() is true. Otherwise falls through to existing path.

Coverage

Bypass handles: id, name, keyword, hypervisor, format, type,
public, featured, bootable, parenttemplateid, accountType,
accountIdIN, zoneid, templateState, removed, onlyReady, pagination,
templatefilter ∈ {self/selfexecutable non-domain-admin, executable,
all admin}.

Falls back to view path: tags, templatefilter ∈ {featured, community,
sharedexecutable, shared, all non-admin, self/selfexecutable for
DOMAIN_ADMIN/RESOURCE_DOMAIN_ADMIN}.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • Build/CI
  • Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

aniishadas and others added 3 commits July 7, 2026 14:38
* added bypass logic to create template_pair from 6 tables, fallback for hard filters

* Removed batching, checking against per pair

* follow try-with-resources design

* unit tests for the bypass logic

* Fix cross-zone template lookup in listTemplates Phase 2

When temp_zone_pair has dcId=0 (cross-zone template with no
data_center), use IS NULL predicate instead of broken EQ/IN
with literal 0 which never matches NULL rows.

* Add filter for non-root domain-admin users

---------

Co-authored-by: anishadas <adas33@apple.com>
Co-authored-by: Aaron Chung <aaron_chung@apple.com>
Co-authored-by: anishadas <adas33@apple.com>
(cherry picked from commit f6c9fd1081f0e1d9386db08e86485b482bf064ab)
@codecov

codecov Bot commented Jul 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 3.69%. Comparing base (a951ac6) to head (21955d9).

❗ There is a different number of reports uploaded between BASE (a951ac6) and HEAD (21955d9). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (a951ac6) HEAD (21955d9)
unittests 1 0
Additional details and impacted files
@@              Coverage Diff              @@
##               4.22   #13566       +/-   ##
=============================================
- Coverage     17.67%    3.69%   -13.98%     
=============================================
  Files          5923      448     -5475     
  Lines        533349    38117   -495232     
  Branches      65248     7067    -58181     
=============================================
- Hits          94253     1409    -92844     
+ Misses       428437    36521   -391916     
+ Partials      10659      187    -10472     
Flag Coverage Δ
uitests 3.69% <ø> (ø)
unittests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants