feat: support hadoop and hbase addons on KubeBlocks 1.1

[baojun-zhang]

Description

This PR adds first-class Hadoop HDFS and HBase support to the KubeBlocks 1.1 addon ecosystem by introducing both addon-level definitions and cluster-level deployment charts.

The change is organized around two capability tracks rather than one-off file additions:

1. HDFS capabilities for standalone and HA deployments on KubeBlocks
2. HBase capabilities for both HDFS-backed cluster mode and local-storage standalone mode

At a high level, this PR contributes:

• New addons/hadoop and addons/hbase packages with ClusterDefinition, ComponentDefinition, ComponentVersion, config templates, startup scripts, config constraints, and release notes
• New addons-cluster/hadoop and addons-cluster/hbase charts for day-1 cluster provisioning
• Runtime wiring for ZooKeeper-based HA coordination, HDFS/HBase service references, storage layouts, probes, and lifecycle handling
• User-facing documentation and values/schema definitions for installing and operating the new addons on KubeBlocks 1.1

Related Issue

N/A

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Refactoring (no functional changes)
• Performance improvement
• Test update

Changes Made

• Added a full Hadoop HDFS addon with both standalone and HA topologies, including component models for hadoop-core, journalnode, namenode, and datanode
• Added a full HBase addon with both distributed cluster mode and standalone mode, including hmaster, hregionserver, and hbase-standalone
• Added cluster charts for HDFS and HBase so users can provision these stacks directly with cluster-scoped values, resource sizing, storage configuration, and external service references

HDFS capabilities

• Introduces two HDFS topologies:
  • standalone: hadoop-core + single namenode + single datanode
  • cluster: hadoop-core + journalnode quorum + dual namenode + scalable datanode
• Adds HA-specific runtime building blocks:
  • JournalNode quorum for shared edits
  • Dual NameNode topology with nn0 and nn1
  • ZKFC sidecar for NameNode automatic failover
  • ZooKeeper service reference wiring for HA election and coordination
• Adds NameNode lifecycle automation:
  • one-time format on ordinal 0
  • bootstrapStandby retry loop for the standby NameNode
  • graceful failover attempt during shutdown before daemon termination
  • ZKFC shutdown coordination through a lifecycle signal file
• Adds DataNode operational behavior for production-style HDFS:
  • hostNetwork / hostPID runtime support
  • per-node data volume mounting
  • decommission flow on preStop
  • hostname-aware registration using pod FQDN
• Adds HDFS configuration rendering for both HA and standalone modes:
  • core-site.xml
  • hdfs-site.xml for NameNode / DataNode / JournalNode
  • hadoop-env.sh
  • log4j.properties
  • hadoop-metrics2.properties
  • include/exclude host lists for decommission control
• Adds day-2 parameterization and validation:
  • replication and max replication
  • NameNode and DataNode handler counts
  • IPC client timeouts and pool sizing
  • permissions and registration checks
  • ZK session timeout, retry count, and parent znode
  • JVM heap sizes for NameNode, DataNode, JournalNode, and ZKFC
• Adds component version mapping for HDFS 3.3.4 and wires dedicated component images for namenode, datanode, and journalnode
• Adds cluster chart values/schema for replicas, storage, service references, ports, resources, HA settings, probes, and update strategy

HBase capabilities

• Introduces two HBase topologies:
  • hbase-cluster: hmaster + hregionserver
  • hbase-standalone: a single-process local-storage HBase deployment
• Adds distributed HBase support on top of HDFS HA:
  • HBase root directory stored in hdfs://<nameservice>/<rootDir>
  • HDFS client-side HA wiring via generated hdfs-site.xml
  • explicit service references to external ZooKeeper and HDFS NameNode services
  • root layout bootstrap in HDFS before HMaster starts
• Adds standalone HBase support decoupled from HDFS:
  • local file://-backed hbase.rootdir
  • persistent volume for standalone data
  • ZooKeeper-only external dependency
• Adds runtime scripts for HBase operations:
  • HMaster startup and graceful shutdown
  • RegionServer startup and graceful unload using RegionMover
  • HDFS root path initialization for cluster mode
  • lightweight process and HTTP-based readiness/liveness checks
• Adds HBase configuration rendering:
  • hbase-site.xml for distributed and standalone modes
  • core-site.xml / hdfs-site.xml injection for HDFS-backed deployments
  • hbase-env.sh
  • HBase metrics and log4j configuration
• Adds operational defaults and tunables:
  • separate JVM sizing for HMaster, RegionServer, and standalone mode
  • JMX ports for HMaster and RegionServer
  • HBase temp/log/PID/data directories
  • HDFS nameservice, HA nodes, RPC/HTTP ports, and replication passthrough
• Adds component version mapping for HBase 2.5.6 and dedicated images for HMaster and RegionServer
• Adds cluster chart values/schema for topology selection, service references, replicas, resources, and standalone storage sizing

Cross-cutting capabilities

• Adds README and release notes for both addons
• Adds chart metadata, helper templates, values, and schema validation for both cluster charts
• Standardizes runtime security context usage and non-root execution defaults across the new components
• Wires KubeBlocks-native ClusterDefinition, ComponentDefinition, ComponentVersion, config templates, and parameter renderers together so the addons are deployable on KubeBlocks 1.1

Testing

Code review coverage for this draft was based on the full 108-file PR content, including addon definitions, cluster charts, config templates, scripts, values, schemas, and release notes.

Recommended validation for this PR:

• Render addons-cluster/hadoop in both standalone and cluster topologies

• Validate NameNode format / standby bootstrap / ZKFC behavior in an HA environment with ZooKeeper

• Validate DataNode registration and preStop decommission behavior under hostNetwork

• Render and deploy addons-cluster/hbase in both distributed and standalone modes

• Validate HBase cluster mode against an HDFS nameservice-backed deployment

• Validate HBase standalone mode with local file:// storage

• I have added tests that prove my fix is effective or that my feature works

• New and existing unit tests pass locally with my changes

• I have tested this on the following platforms:

  • Linux
  • macOS
  • Windows

Checklist

• My code follows the project's coding style
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• Any dependent changes have been merged and published

Screenshots (if applicable)

N/A

Additional Notes

• This PR is not just adding charts; it introduces two new addon stacks end to end:
  • Hadoop HDFS addon + cluster chart
  • HBase addon + cluster chart
• HDFS support is broader than a static HA manifest. The PR includes runtime logic for NameNode initialization, standby bootstrap, ZKFC startup, JournalNode-based shared edits, readiness checks, and DataNode decommission handling.
• HBase support is broader than a simple component wrapper. The PR explicitly covers:
  • distributed mode on top of HDFS + ZooKeeper
  • standalone mode on top of local storage + ZooKeeper
  • bootstrap logic for HDFS root layout
  • graceful RegionServer shutdown with region unload
• The HBase distributed path assumes an external HDFS deployment exposed through service references and nameservice-based addressing.
• The HBase standalone path is intentionally decoupled from HDFS and uses file:// storage semantics.
• Manual integration testing is still important for HA failover semantics, shutdown ordering, and HBase-on-HDFS interoperability under real cluster conditions.

[leon-ape]

Findings:

• addons-cluster/hbase/templates/cluster.yaml renders spec.topology: cluster, while addons/hbase/templates/clusterdefinition.yaml only defines hbase-cluster and hbase-standalone.
• The default HBase cluster renders serviceRefs.hdfs-namenode...port: client, but the HDFS namenode ComponentDefinition exposes service ports fs and http.
• The new Hadoop addon introduces ParamConfigRenderer resources in addons/hadoop/templates/pcr-*.yaml; PCR is deprecated for the current addon API.
• This PR adds new Hadoop/HBase addons and cluster charts, but the PR has no provision and service-connectivity validation evidence.
• pr-label-check is failing because the PR only has chart-release and no pick/nopick/pick-* label.

[baojun-zhang]

evidence of run success :

[baojun-zhang]

Metrics:

HDFS

HBASE