📦 Hex Terrains Framework – Data Layer System

Overview

The Hex Terrains Framework features a robust, job-safe data layer system for managing large-scale hexagonal terrain data using Unity DOTS. Each terrain is an entity composed of one or more TerrainLayers, which encapsulate specific data (e.g., surface, items, areas) in a structured, extensible format. At the core of each TerrainLayer is a DataLayer, a highly-performant container with built-in support for job dependencies, chunk-based dirty tracking, and safe disposal.

🧱 What is a DataLayer?

A DataLayer is the foundational building block for storing and managing layer-specific data across your terrain. It:

  • Stores data in managed or unmanaged structures
  • Tracks read/write JobHandle dependencies for safety and performance
  • Supports immediate and deferred read/write access
  • Tracks dirty states and change versions for efficient updates
  • Can be extended for arrays, lists, maps, or custom structures

Each DataLayer is either:

  • Flat – inherited from DataLayer, tracks global state (e.g., ArrayDataLayer, ListDataLayer)
  • Chunked – inherited from ChunkedDataLayer, tracks dirty state on a per-chunk basis (e.g., ArrayChunkedDataLayer, NativeListChunkedDataLayer)

🧬 DataLayer Types

Flat Data Layers (DataLayer)

These layers manage data in simple structures such as arrays or lists, without spatial chunk tracking:

Class Name Description
ArrayDataLayer<T> Backed by a managed array
ListDataLayer<T> Backed by a managed list
NativeArrayDataLayer<T> Uses NativeArray<T>
NativeListDataLayer<T> Uses NativeList<T>
DictionaryDataLayer<TKey, TValue> Uses managed dictionary
NativeHashMapDataLayer<TKey, T> Uses NativeHashMap
NativeParallelHashMapDataLayer Uses NativeParallelHashMap
... More variants for hash sets and multi-maps

All these layers implement a common pattern:

  • Init(), GetData(), SetData()
  • TryGetData(), TrySetData()
  • DisposeAllItems(), DisposeExcessItems()

Chunked Data Layers (ChunkedDataLayer)

These layers extend DataLayer with dirty chunk tracking, useful for marking and updating only modified portions of a terrain.

Class Name Description
ArrayChunkedDataLayer<T> Managed array with chunk tracking
ListChunkedDataLayer<T> Managed list with chunk tracking
NativeArrayChunkedDataLayer<T> Unmanaged array (NativeArray)
NativeListChunkedDataLayer<T> Unmanaged list (NativeList)

Chunked layers track:

  • ChunkDirtyGrid: flags for which chunks are dirty
  • DirtyChunks: hash set of indices that need update
  • ChunkSize, ChunkGridSize: defines how data is divided spatially

They also provide methods like:

  • SetCellDirty(cellIndex, checkEdges)
  • SetAllChunksDirty(), ClearAllChunksDirty()
  • MergeChunkDirtyGrids(...) – for combining dirty flags from multiple layers

🔁 Job Safety and Dependency Management

All layers support Unity's JobHandle dependency tracking to ensure safety when scheduling or accessing data.

Usage Patterns

Operation Method Notes
Immediate Read OpenToRead() Completes write jobs
Immediate Write OpenToWrite() Completes read and write jobs
Schedule Read Job PrepareToRead() Use returned JobHandle as dependency
Schedule Write Job PrepareToWrite() Use returned JobHandle as dependency
Register Job Completion AddRead/WriteDependency(JobHandle) Used in systems

These mechanisms allow multiple systems and jobs to work with terrain layers without risking race conditions or data corruption.

♻️ Disposal and Cleanup

If a DataLayer holds items that implement IDisposable, it will automatically call .Dispose() when resized or disposed.

  • IsDisposableItems: flag to enable per-item disposal
  • DisposeExcessItems(int newSize): disposes unused items when shrinking
  • DisposeAllItems(): disposes everything
  • Dispose(): cleans up all resources and dependencies

🧩 Extending DataLayers

To add your own custom layer type:

  1. Inherit from DataLayer or ChunkedDataLayer
  2. Implement required properties like Length, and override Init, GetData, SetData
  3. Use appropriate container (List<T>, NativeArray<T>, etc.)
  4. Optionally implement chunk tracking if spatial change detection is needed