Config with ~1 Line Blitz

🌱

ZhiJian is an unifying and rapidly deployable toolbox for pre-trained model reuse.

  • What & Why Reuse?

    • Performing downstream tasks with the help of pre-trained model, including model structures, weights, or other derived rules.

    • Significantly accelerating convergence and improving downstream performance.

In ZhiJian, adding the LoRA module to the pre-trained model and adjusting which part of the parameters to fine-tune just require about :customlyellow:`one` line of code.

Overview

In the following example, we show how ZhiJian:

  • Represent the modules of the pre-trained model

  • Config the extended add-in module with entry points

Modules of Pre-trained Model in One Line description

In the Architect module, to facilitate the modification of model structures, additional adaptive structures are incorporated into pre-trained models. ZhiJian accepts a one-line serialized representation of the base pre-trained model, as exemplified in the Vision Transformer model from the timm library in the following manner:

../_images/tutorial_one_line_config.png

The modules within the parentheses () represent the base pre-trained model, and the dot . is used as a access operator.

The arrows -> indicate the connections between modules, and ellipsis ... represents default modules. Partial structures can be connected with arrows.

Extended Add-in Module with Entry Points

We use (): ` to denote an additional adaptive structure, where the part after the dot :code:.` represents the main forward function of the extra structure. The data flows into the module and primarily passes through this method.

We use {} to indicate the entry points of the extra structure into the pre-trained model, encompassing the entry of source model features and the return points of features after the added structure is processed.

With the aforementioned configuration, ZhiJian seamlessly supports the modification of pre-trained model structures. It automatically recognizes the additional structures defined in zhijianmodelsaddin, enabling the construction of pre-trained models.