Add VendorModuleService for Composer-based Webtrees modules

[magicsunday]

Add VendorModuleService for Composer-based Webtrees modules

🚀 Feature: Vendor Module Support via Composer (VendorModuleService)

Overview

This PR introduces a new service class, VendorModuleService, that allows Webtrees to seamlessly load modules installed via Composer.

A key part of this feature is the introduction of a new Composer package type: webtrees-module. This package type signals that a Composer package is a Webtrees extension, making it automatically discoverable and loadable by the Webtrees core.

This modernizes module management and brings Webtrees closer in line with practices used by frameworks such as Symfony, Laravel, and TYPO3.

Why?

Currently, Webtrees modules are installed and managed manually in the modules_v4 directory. While functional, this approach has limitations:

• ❌ No standardized distribution format for third-party modules
• ❌ Manual installation and updates are error-prone
• ❌ No integration with Composer’s dependency management

By enabling Composer-managed modules (via webtrees-module), we provide:

• ✅ A standardized installation process (composer require vendor/package)
• ✅ Automatic updates alongside Webtrees itself
• ✅ A cleaner separation of first-party and third-party modules
• ✅ A familiar workflow for PHP developers who already rely on Composer

This makes Webtrees more attractive to external developers and lowers the barrier for creating and sharing extensions.

How It Works

The VendorModuleService leverages Composer’s InstalledVersions API to:

1. Discover installed Composer packages of type webtrees-module
2. Load and validate their module.php entrypoint
3. Register the module within Webtrees’ existing ModuleService
4. Provide meaningful error handling via FlashMessages

Internally:

• Modules are identified by composer.json → "type": "webtrees-module"
• Installed paths are retrieved via Composer’s runtime API
• Modules are assigned unique names based on their package (_mypackage_)

Benefits

• For Developers: Publish Webtrees modules directly on Packagist, making them discoverable and installable with a single command.
• For Administrators: No need to manually upload files; Composer ensures correct versions and resolves dependencies.
• For the Project: Aligns Webtrees with modern PHP ecosystem practices, ensuring sustainability and extensibility.

📦 Example composer.json for a Webtrees Module

{
  "name": "my-vendor/my-webtrees-module",
  "description": "An example Webtrees module distributed via Composer.",
  "type": "webtrees-module",
  "license": "GPL-3.0-or-later",
  "authors": [
    {
      "name": "Your Name",
      "email": "you@example.com"
    }
  ],
  "require": {
    "php": ">=8.1",
    "fisharebest/webtrees": "^2.2"
  },
  "autoload": {
    "psr-4": {
      "MyVendor\\MyWebtreesModule\\": "src/"
    }
  }
}

[magicsunday]

Together with Webtrees 2.2.4 or dev-main, this can currently be tested using the magicsunday/webtrees-fan-chart package, for example. To do this, the WIP branch must be checked out.

composer require magicsunday/webtrees-fan-chart:dev-WIP

[codecov]

Codecov Report

❌ Patch coverage is 56.16438% with 32 lines in your changes missing coverage. Please review.
✅ Project coverage is 36.88%. Comparing base (a64cc99) to head (65ccb57).
⚠️ Report is 123 commits behind head on main.

Files with missing lines	Patch %	Lines
app/Services/Composer/VendorModuleService.php	54.28%	32 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #5227      +/-   ##
============================================
+ Coverage     35.49%   36.88%   +1.39%     
- Complexity    11211    11554     +343     
============================================
  Files          1166     1208      +42     
  Lines         48077    49796    +1719     
============================================
+ Hits          17065    18369    +1304     
- Misses        31012    31427     +415     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[kiwi3685]

I develop modules purely for my own use. I have no need or interest in using composer. I know there are many others doing the same. For us modules_v4 is simple and works well.

So how does this proposal affect us? There is no mention of users like us in this.

[magicsunday]

Thanks for raising this — it’s an important point.

To be clear: this proposal does not remove or break the existing modules_v4 system. If you are developing modules purely for your own use, or you prefer to keep things simple by dropping files into modules_v4, you can continue to do so exactly as you always have. Nothing changes for you.

What this PR does is add an additional option for distributing and installing modules via Composer, using the new webtrees-module package type. This is mainly targeted at:

• Developers who want to publish their modules to Packagist for broader use
• Administrators who prefer dependency management through Composer
• The Webtrees ecosystem as a whole, to provide a standardized, modern way of sharing extensions

So to summarize:

• ✅ Your existing workflow with modules_v4 continues to be fully supported
• ✅ No existing modules will need to be rewritten or migrated
• ✅ Composer-based modules are an extra path, not a replacement

The goal here is to expand possibilities without taking anything away.

[magicsunday]

Hi @fisharebest, I hope you're well. I'd appreciate it if you'd take a look at this PR. If you find any issues, I'm happy to adjust or improve it. Thank you for your time and feedback!

[ekdahl]

Nice work!
As a module developer, what steps are needed to publish a module, more than adding the JSON file like in your example?
Does this add more dependencies (Composer) for webtrees, or is Composer always installed when I have PHP in for example Debian?

[magicsunday]

Thanks for the kind feedback! Let me clarify:

What steps are needed to publish a module?

• You create a composer.json file for your module with "type": "webtrees-module".
• Add your module code (with module.php as the entrypoint).
• Push your code to a public Git repository (e.g. GitHub).
• Submit it to Packagist.org (which is essentially “the Composer directory”) so that users can install it with a simple composer require vendor/package.

That’s it — Composer takes care of the rest (autoloading, versioning, installation path, etc.).

Does this add more dependencies for webtrees?
No new dependencies are introduced in Webtrees itself. Composer support is already bundled with Webtrees’ development workflow.

• If you install Webtrees from the official release .zip, nothing changes — you don’t need Composer at all.
• If you install Webtrees via composer create-project, you already have Composer on your system.
• On platforms like Debian, Composer is not installed automatically with PHP — but admins who want to install Composer-based modules can install Composer once (apt install composer or via the Composer installer).

So in short:

• ✅ No extra requirement for existing users.
• ✅ Composer is only needed if you want to use Composer-based modules.
• ✅ Module developers benefit from the familiar Composer workflow.

[BertKoor]

As I understand it, this approach will complicate things tremendously for site admins that 'simply' want to install or try out a vendor module.

Currently all that's needed is unzip the module source code into a subdirectory of modules_v4 on their server.

With this mechanism they need to clone the webtrees git repo, edit some file, have a local PHP development environment in order to execute composer install and then upload whatever that has produced to their server.

Thus the requirements for using such modules are raised significantly. Or am I missing something?

[kiwi3685]

Hi @BertKoor
I asked a similar question when this was first presented here (see earliest comment above).
For my purposes this was answered very satisfactorily.
But you should review the answer though, as you may be thinking of something I missed.

[BertKoor]

Thanks for pointing that out, @kiwi3685 . I thought I had read the complete discussion, but apparently I had misunderstood some nuances. You addressed concerns from the point of view of a module developer. It was mentioned that this "does not remove or break the existing modules_v4 system" and I understood that as implying only a simple backwards-compatibility guarantee. There was no explicit mention of impact on site administrators.

Now should I understand that "composer"-ised modules can be installed the old way as well? In my experience more options leads to confusion, especially on unknown territory. And I hope that n>1 for this feature, but I have my doubts.

[magicsunday]

• The existing modules_v4 system remains fully supported — you can still just unzip and use modules as before.

• Modules can still be distributed as ZIP files and installed manually; Composer is only needed if you choose to use it.

• Composer support is an additional option, primarily aimed at:

  • Developers who publish their modules on Packagist (like me)
  • Power users or automated environments where dependency management and versioning via Composer are desirable (like me => https://github.com/magicsunday/webtrees-docker/)
    • My NAS doesn't allow me to install additional packages like PHP, etc. Therefore, I've put together a Docker development environment that makes me completely independent of the host OS.
    • With Docker, I package my development environment into a self-contained "box." I can run any PHP or database version I want without touching my NAS/host system. The box is reproducible (everyone gets exactly the same environment with docker compose up), isolated, and can be removed at any time without leaving any residue. This way, I avoid "only runs on me" issues, keep the host clean, and can quickly test or share production-level setups.

@fisharebest
Have you had a chance to take a look at it yet?

[fisharebest]

Instead of modifying webtrees to search for modules in a new location (vendor/...), would it be simpler to configure the modules to install in the target folder modules_v4, instead of vendor?

From reading the composer documentation, I think that the following steps are required

1. Add this section to composer.json in webtrees.

{
  "extra": {
    "installer-paths": {
      "modules_v4": ["type:webtrees-module"]
    }
  }
}

If you want to script this, you can use this command line to make the edit.

jq '.extra["installer-paths"]["modules_v4"] = ["type:webtrees-module"]' composer.json > tmp && mv tmp composer.json

2. In the module, add "type": "webtrees-module" to it's composer.json. I see that magicsunday/webtrees-fan-chart has already done this.

3. In the module add composer require composer/installers.

On the webtrees system, you should be able to run composer require magicsunday/webtrees-fan-chart:dev-WIP and it will be installed in modules_v4.

Does this work?

[fisharebest]

/edit

The installation path might need to be "modules_v4/{$name}" instead of "modules_v4". It's not clear from the documentation.

[fisharebest]

I was wondering if composer/installers could be installed in webtrees (i.e. only once), rather than in every module. But I couldn't make it work.

[fisharebest]

I have created a minimal webtrees module installer. It's just a few lines of code.

I see that @magicsunday has created a much more comprehensive installer.

I didn't look to see what additional features it provides. But a module should be able to use any installer, not just one of these two.

Now, if a module specifies its type ("type": "webtrees-module") and requires an installer in its composer.json, then it can be installed using composer.

e.g. this is an example module that you can install now.

composer require webtrees/example-module dev-main

[magicsunday]

Hi @fisharebest, thank you so much for looking into this and creating the minimal installer – I really appreciate your effort!

I'd like to share my thoughts on why I believe the VendorModuleService approach offers significant technical advantages over the installer-based approach. To clarify, this isn't a core change, but rather a core extension. The existing code remains unchanged; only an additional path for module discovery is added.

A real-world use case: Development workflow. Developing and maintaining multiple interdependent modules using the modules_v4 approach is cumbersome in practice. My modules (fan-chart, pedigree-chart, descendants-chart) all depend on a common base package (webtrees-module-base). With modules_v4, any change to the base package requires manually copying or creating symbolic links – and synchronizing all modules is prone to errors.

With the VendorModuleService, Composer manages the entire dependency graph automatically. A composer update loads the correct version of each module and its dependencies into the vendor/ directory, making it immediately ready for use. No manual file moving, no broken symbolic links, no version conflicts.

You can see this in action in my project https://github.com/magicsunday/webtrees-docker. There, the entire Webtrees stack (including custom modules) is managed through a single composer.json file. This works seamlessly only because modules and their dependencies reside in the vendor/ directory, where Composer expects them to be.

Established pattern in the PHP ecosystem: This approach follows the pattern used by major frameworks. For example, TYPO3 uses the package type typo3-cms-extension – and in Composer mode, the core delegates the search for extensions to Composer's own mechanisms. Extensions remain in the vendor/ directory and are discovered via Composer's runtime API. The VendorModuleService follows this exact principle, using webtrees-module as the package type.

1. Composer-native behavior:
   With the VendorModuleService, modules remain in the vendor/ directory, where Composer manages them natively. The installation approach physically moves files to modules_v4/, which can lead to problems in various scenarios—for example, file permission issues, interrupted updates, symlink configurations, or in container environments.

2. PSR-4 autoloading works by default.
   Packages in vendor/ are automatically loaded correctly via Composer's autoload.php using PSR-4 autoloading. Modules moved to modules_v4/ are decoupled from the Composer autoloader and therefore require a workaround: registering their own class loader in modules.php:

$loader = new ClassLoader();
$loader->addPsr4('MagicSunday\\Webtrees\\FanChart\\', __DIR__ . '/src');
$loader->register();

With the VendorModuleService, this workaround is unnecessary, as the Composer autoloader already knows the module's namespace.

3. Dependency Resolution is Preserved
   If a module in vendor/ has its own Composer dependencies—such as my chart modules, which depend on webtrees-module-base—everything is resolved correctly. With the installer approach, the module itself is moved to modules_v4/, but its dependencies remain in vendor/. This separation can lead to problems with autoloading and paths, especially if the dependency tree is more than one level deep.

4. No mandatory installer dependency per module
   The installer approach requires each individual module to declare a dependency on an installer package. With the VendorModuleService, a module only needs to include "type": "webtrees-module" in its composer.json – detection is handled by the kernel.

5. Clear separation of responsibilities
   With the VendorModuleService, there is a clear boundary: vendor/ is managed by Composer, while modules_v4/ is managed manually. The installation approach mixes both in modules_v4/, which can lead to conflicts – for example, if someone manually deletes a module installed with Composer while it is still present in composer.lock.

I am happy to adapt the implementation if you have concerns about specific code sections. However, I am convinced that keeping the modules in vendor/ and utilizing Composer's features is the more robust and future-proof solution.

[magicsunday]

Hi @fisharebest, following up here — I think the discussion has surfaced both approaches clearly, so I'd rather not restate the technical arguments again.

Where we are, briefly:

• Your preference: keep core unchanged and let modules install into modules_v4/ via an installer (your minimal one, or a more comprehensive one).
• This PR: an additive core extension — modules_v4/ is untouched and keeps working exactly as today; it only adds a second discovery path so Composer-managed modules can stay in vendor/, where their dependency graph and PSR-4 autoloading already live. The two can coexist; this doesn't remove the installer route.

What would really help is a direction decision, so the PR doesn't sit open indefinitely:

• If you'd consider taking vendor/-based discovery into core — now, or with any changes you'd like — I'm glad to do the work.
• If you'd rather keep core out of it and standardise on the installer approach, just say the word and I'll close this, no hard feelings; I'll keep maintaining the service downstream for my own stack.

Either outcome is fine — I'd just like to know which direction to invest in. Thanks for your time on this.