Models factory

Models

Model and template format directory that unifies (and simplifies) model access and configuration.

This env variable is checked and used during initialization

SIBILA_MODELS: ';'-delimited list of folders where to find: models.json, formats.json and model files.

= Models Directory =

Useful to create models from resource names like "llamacpp:openchat" or "openai:gpt-4". This makes it simple to change a model, store model settings, to compare model outputs, etc.

User can add new entries from script or with JSON filenames, via the add() call. New directory entries with the same name are merged into existing ones for each added config.

Uses file "sibila/res/base_models.json" for the initial defaults, which the user can augment by calling setup() with own config files or directly adding model config with set_model().

An example of a model directory JSON config file:

{
    # "llamacpp" is a provider, you can then create models with names 
    # like "provider:model_name", for ex: "llamacpp:openchat"
    "llamacpp": { 

        "_default": { # place here default args for all llamacpp: models.
            "genconf": {"temperature": 0.0}
            # each model entry below can then override as needed
        },

        "openchat": { # a model definition
            "name": "openchat-3.5-1210.Q4_K_M.gguf",
            "format": "openchat" # chat template format used by this model
        },

        "phi2": {
            "name": "phi-2.Q4_K_M.gguf", # model filename
            "format": "phi2",
            "genconf": {"temperature": 2.0} # a hot-headed model
        },

        "oc": "openchat" 
        # this is a link: "oc" forwards to the "openchat" entry
    },

    # The "openai" provider. A model can be created with name: "openai:gpt-4"
    "openai": { 

        "_default": {}, # default settings for all OpenAI models

        "gpt-3.5": {
            "name": "gpt-3.5-turbo-1106" # OpenAI's model name
        },

        "gpt-4": {
            "name": "gpt-4-1106-preview"
        },
    },

    # "alias" entry is not a provider but a way to have simpler alias names.
    # For example you can use "alias:develop" or even simpler, just "develop" to create the model:
    "alias": { 
        "develop": "llamacpp:openchat",
        "production": "openai:gpt-3.5"
    }
}

Rules for entry inheritance/overriding

Entries in the '_default' key of each provider will serve as defaults for models of that provider. Model entries in base_models_dir (automatically loaded from 'res/base_models.json') are overridden by any entries of the same name loaded from a local 'models.json' file with Models.setup(). Here, overridden means local keys of the same name replace base keys (as a dict.update()). However '_default' entries only apply separately to either base_models_dir or 'local models.json', as in a lexical scope.

= Format Directory =

Detects chat templates from model name/filename or uses from metadata if possible.

This directory can be setup from a JSON file or by calling set_format().

Any new entries with the same name replace previous ones on each new call.

Initializes from file "sibila/res/base_formats.json".

Example of a "formats.json" file:

{
    "chatml": {
        # template is a Jinja template for this model
        "template": "{% for message in messages %}..."
    },

    "openchat": {
        "match": "openchat", # a regexp to match model name or filename
        "template": "{{ bos_token }}..."
    },    

    "phi": {
        "match": "phi",
        "template": "..."
    },

    "phi2": "phi",
    # this is a link: "phi2" -> "phi"
}

Jinja2 templates receive a standard ChatML messages list (created from a Thread) and must deal with the following:

• In models that don't use a system message, template must take care of prepending it to first user message.

• The add_generation_prompt template variable is always set as True.

setup classmethod

setup(
    path=None, clear=False, add_cwd=True, load_from_env=True
)

Initialize models and formats directory from given model files folder and/or contained configuration files. Path can start with "~/" current account's home directory.

Parameters:

Name	Type	Description	Default
path	Optional[Union[str, list[str]]]	Path to a folder or to "models.json" or "formats.json" configuration files. Defaults to None which tries to initialize from defaults and env variable.	None
clear	bool	Set to clear existing directories before loading from path arg.	False
add_cwd	bool	Add current working directory to search path.	True
load_from_env	bool	Load from SIBILA_MODELS env variable?	True

Source code in sibila/models.py

@classmethod
def setup(cls,
          path: Optional[Union[str,list[str]]] = None,
          clear: bool = False,
          add_cwd: bool = True,
          load_from_env: bool = True):
    """Initialize models and formats directory from given model files folder and/or contained configuration files.
    Path can start with "~/" current account's home directory.

    Args:
        path: Path to a folder or to "models.json" or "formats.json" configuration files. Defaults to None which tries to initialize from defaults and env variable.
        clear: Set to clear existing directories before loading from path arg.
        add_cwd: Add current working directory to search path.
        load_from_env: Load from SIBILA_MODELS env variable?
    """

    if clear:
        cls.clear()

    cls._ensure(add_cwd, 
                load_from_env)

    if path is not None:
        if isinstance(path, str):
            path_list = [path]
        else:
            path_list = path

        cls._read_any(path_list)

create classmethod

create(
    res_name,
    genconf=None,
    ctx_len=None,
    *,
    resolved_create_args=None,
    **over_args
)

Create a model.

Parameters:

Name	Type	Description	Default
res_name	str	Resource name in the format: provider:model_name, for example "llamacpp:openchat".	required
genconf	Optional[GenConf]	Optional model generation configuration. Overrides set_genconf() value and any directory defaults. Defaults to None.	None
ctx_len	Optional[int]	Maximum context length to be used. Overrides directory defaults. Defaults to None.	None
resolved_create_args	Optional[dict]	Pass an empty dict to be filled by this method with the resolved args used in model creation. Defaults to None.	None
over_args	Union[Any]	Model-specific creation args, which will override default args set in model directory.	{}

Returns:

Name	Type	Description
Model	Model	the initialized model.

Source code in sibila/models.py

@classmethod
def create(cls,
           res_name: str,

           # common to all providers
           genconf: Optional[GenConf] = None,
           ctx_len: Optional[int] = None,

           *,
           # debug/testing
           resolved_create_args: Optional[dict] = None,

           # model-specific overriding:
           **over_args: Union[Any]) -> Model:
    """Create a model.

    Args:
        res_name: Resource name in the format: provider:model_name, for example "llamacpp:openchat".
        genconf: Optional model generation configuration. Overrides set_genconf() value and any directory defaults. Defaults to None.
        ctx_len: Maximum context length to be used. Overrides directory defaults. Defaults to None.
        resolved_create_args: Pass an empty dict to be filled by this method with the resolved args used in model creation. Defaults to None.
        over_args: Model-specific creation args, which will override default args set in model directory.

    Returns:
        Model: the initialized model.
    """

    try:
        provider, _, args = cls.resolve_model_entry(res_name, **over_args)
    except ValueError as e:
        raise NameError(str({e}))

    # override genconf, ctx_len
    if genconf is None:
        genconf = cls.genconf

    if genconf is not None:
        args["genconf"] = genconf

    elif "genconf" in args and isinstance(args["genconf"], dict):
        # transform dict into a GenConf instance:
        args["genconf"] = GenConf.from_dict(args["genconf"])

    if ctx_len is not None:
        args["ctx_len"] = ctx_len

    if resolved_create_args is not None:
        resolved_create_args.update(args)


    logger.debug(f"Resolved '{res_name}' to provider '{provider}' with args: {args}")


    model: Model
    if provider == "anthropic":

        from .anthropic import AnthropicModel
        model = AnthropicModel(**args)

    elif provider == "fireworks":

        from .schema_format_openai import FireworksModel
        model = FireworksModel(**args)

    elif provider == "groq":

        from .schema_format_openai import GroqModel
        model = GroqModel(**args)

    elif provider == "llamacpp":
        from .llamacpp import LlamaCppModel, extract_sub_paths

        # resolve filename -> path. Path filenames can be in the form model1*model2
        sub_paths = args["name"].split('*')

        # only resolve first path. If not found, let LlamaCpp raise the error below
        sub_paths[0] = cls._locate_file(sub_paths[0]) or sub_paths[0]

        # rejoin located paths with '*' (if multiple)
        path = '*'.join(sub_paths)
        logger.debug(f"Resolved llamacpp model '{args['name']}' to '{path}'")

        # rename "name" -> "path" which LlamaCppModel is expecting
        del args["name"]
        args["path"] = path

        model = LlamaCppModel(**args)

    elif provider == "mistral":

        from .mistral import MistralModel
        model = MistralModel(**args)

    elif provider == "openai":

        from .openai import OpenAIModel
        model = OpenAIModel(**args)

    elif provider == "together":

        from .schema_format_openai import TogetherModel
        model = TogetherModel(**args)

    else:
        raise ValueError(f"Unknown provider '{provider}' for '{res_name}'")



    return model

add_models_search_path classmethod

add_models_search_path(path)

Prepends new paths to model files search path.

Parameters:

Name	Type	Description	Default
path	Union[str, list[str]]	A path or list of paths to add to model search path.	required

Source code in sibila/models.py

@classmethod
def add_models_search_path(cls,
                           path: Union[str,list[str]]):
    """Prepends new paths to model files search path.

    Args:
        path: A path or list of paths to add to model search path.
    """

    cls._ensure()

    prepend_path(cls.models_search_path, path)

    logger.debug(f"Adding '{path}' to search_path")

set_genconf classmethod

set_genconf(genconf)

Set the GenConf to use as default for model creation.

Parameters:

Name	Type	Description	Default
genconf	GenConf	Model generation configuration.	required

Source code in sibila/models.py

@classmethod
def set_genconf(cls,
                genconf: GenConf):
    """Set the GenConf to use as default for model creation.

    Args:
        genconf: Model generation configuration.
    """
    cls.genconf = genconf

list_models classmethod

list_models(
    name_query, providers, include_base, resolved_values
)

List format entries matching query.

Parameters:

Name	Type	Description	Default
name_query	str	Case-insensitive substring to match model names. Empty string for all.	required
providers	list[str]	Filter by these exact provider names. Empty list for all.	required
include_base	bool	Also list fused values from base_models_dir.	required
resolved_values	bool	Return resolved entries or raw ones.	required

Returns:

Type	Description
dict	A dict where keys are model res_names and values are respective entries.

Source code in sibila/models.py

@classmethod
def list_models(cls,
                name_query: str,
                providers: list[str],
                include_base: bool,
                resolved_values: bool) -> dict:
    """List format entries matching query.

    Args:
        name_query: Case-insensitive substring to match model names. Empty string for all.
        providers: Filter by these exact provider names. Empty list for all.
        include_base: Also list fused values from base_models_dir.
        resolved_values: Return resolved entries or raw ones.

    Returns:
        A dict where keys are model res_names and values are respective entries.
    """

    cls._ensure()

    models_dir = cls.fused_models_dir() if include_base else cls.models_dir

    out = {}

    name_query = name_query.lower()

    for prov_name in models_dir:

        if providers and prov_name not in providers:
            continue

        prov_dic = models_dir[prov_name]

        for name in prov_dic:

            if name == cls.DEFAULT_ENTRY_NAME:
                continue

            if name_query and name_query not in name.lower():
                continue

            entry_res_name = prov_name + ":" + name

            if resolved_values:
                # okay to use get_model_entry() because it resolves to fused
                res = cls.get_model_entry(entry_res_name) # type: ignore[assignment]
                if res is None:
                    continue
                else:
                    val = res[1]
            else:
                val = prov_dic[name]

            out[entry_res_name] = val

    return out

get_model_entry classmethod

get_model_entry(res_name)

Get a resolved model entry. Resolved means following any links.

Parameters:

Name	Type	Description	Default
res_name	str	Resource name in the format: provider:model_name, for example "llamacpp:openchat".	required

Returns:

Type	Description
Union[tuple[str, dict], None]	Resolved entry (res_name,dict) or None if not found.

Source code in sibila/models.py

@classmethod
def get_model_entry(cls,
                    res_name: str) -> Union[tuple[str,dict],None]:
    """Get a resolved model entry. Resolved means following any links.

    Args:
        res_name: Resource name in the format: provider:model_name, for example "llamacpp:openchat".

    Returns:
        Resolved entry (res_name,dict) or None if not found.
    """

    cls._ensure()        

    models_dir = cls.fused_models_dir()

    # resolve "alias:name" res names, or "name": "link_name" links
    provider,name = resolve_model(models_dir, res_name, cls.ALL_PROVIDER_NAMES)
    # arriving here, prov as a non-link dict entry
    logger.debug(f"Resolved model '{res_name}' to '{provider}','{name}'")

    prov = models_dir[provider]

    if name in prov:
        return provider + ":" + name, prov[name]
    else:
        return None

has_model_entry classmethod

has_model_entry(res_name)

Source code in sibila/models.py

@classmethod
def has_model_entry(cls,
                    res_name: str) -> bool:
    return cls.get_model_entry(res_name) is not None

set_model classmethod

set_model(
    res_name, model_name, format_name=None, genconf=None
)

Add model configuration for given res_name.

Parameters:

Name	Type	Description	Default
res_name	str	A name in the form "provider:model_name", for example "openai:gtp-4".	required
model_name	str	Model name or filename identifier.	required
format_name	Optional[str]	Format name used by model. Defaults to None.	None
genconf	Optional[GenConf]	Base GenConf to use when creating model. Defaults to None.	None

Raises:

Type	Description
ValueError	If unknown provider.

Source code in sibila/models.py

@classmethod
def set_model(cls,
              res_name: str,
              model_name: str,
              format_name: Optional[str] = None,
              genconf: Optional[GenConf] = None):
    """Add model configuration for given res_name.

    Args:
        res_name: A name in the form "provider:model_name", for example "openai:gtp-4".
        model_name: Model name or filename identifier.
        format_name: Format name used by model. Defaults to None.
        genconf: Base GenConf to use when creating model. Defaults to None.

    Raises:
        ValueError: If unknown provider.
    """

    cls._ensure()

    provider,name = provider_name_from_urn(res_name, False)
    if provider not in cls.ALL_PROVIDER_NAMES:
        raise ValueError(f"Unknown provider '{provider}' in '{res_name}'")

    entry: dict = {
        "name": model_name
    }

    if format_name:
        if not cls.has_format_entry(format_name):
            raise ValueError(f"Could not find format '{format_name}'")
        entry["format"] = format_name

    if genconf:
        entry["genconf"] = genconf.as_dict()

    cls.models_dir[provider][name] = entry

update_model classmethod

update_model(
    res_name,
    model_name=None,
    format_name=None,
    genconf=None,
)

update model fields

Parameters:

Name	Type	Description	Default
res_name	str	A name in the form "provider:model_name", for example "openai:gtp-4".	required
model_name	Optional[str]	Model name or filename identifier. Defaults to None.	None
format_name	Optional[str]	Format name used by model. Use "" to delete. Defaults to None.	None
genconf	Union[GenConf, str, None]	Base GenConf to use when creating model. Defaults to None.	None

Raises:

Type	Description
ValueError	If unknown provider.

Source code in sibila/models.py

@classmethod
def update_model(cls,
                 res_name: str,
                 model_name: Optional[str] = None,
                 format_name: Optional[str] = None,
                 genconf: Union[GenConf,str,None] = None):

    """update model fields

    Args:
        res_name: A name in the form "provider:model_name", for example "openai:gtp-4".
        model_name: Model name or filename identifier. Defaults to None.
        format_name: Format name used by model. Use "" to delete. Defaults to None.
        genconf: Base GenConf to use when creating model. Defaults to None.

    Raises:
        ValueError: If unknown provider.
    """

    cls._ensure()

    provider,name = provider_name_from_urn(res_name, False)
    if provider not in cls.ALL_PROVIDER_NAMES:
        raise ValueError(f"Unknown provider '{provider}' in '{res_name}'")

    entry = cls.models_dir[provider][name]

    if model_name:
        entry["name"] = model_name

    if format_name is not None:
        if format_name != "":
            if not cls.has_format_entry(format_name):
                raise ValueError(f"Could not find format '{format_name}'")
            entry["format"] = format_name
        else:
            del entry["format"]

    if genconf is not None:
        if genconf != "":
            entry["genconf"] = genconf
        else:
            del entry["genconf"]

set_model_link classmethod

set_model_link(res_name, link_name)

Create a model link into another model.

Parameters:

Name	Type	Description	Default
res_name	str	A name in the form "provider:model_name", for example "openai:gtp-4".	required
link_name	str	Name of model this entry links to.	required

Raises:

Type	Description
ValueError	If unknown provider.

Source code in sibila/models.py

@classmethod
def set_model_link(cls,
                   res_name: str,
                   link_name: str):
    """Create a model link into another model.

    Args:
        res_name: A name in the form "provider:model_name", for example "openai:gtp-4".
        link_name: Name of model this entry links to.

    Raises:
        ValueError: If unknown provider.
    """

    cls._ensure()

    provider,name = provider_name_from_urn(res_name, True)
    if provider not in cls.ALL_PROVIDER_NAMES:
        raise ValueError(f"Unknown provider '{provider}' in '{res_name}'")

    # first: ensure link_name is a res_name
    if ':' not in link_name:
        link_name = provider + ":" + link_name

    if not cls.has_model_entry(link_name):
        raise ValueError(f"Could not find linked model '{link_name}'")

    # second: check link name is without provider if same
    link_split = link_name.split(":")
    if len(link_split) == 2:
        if link_split[0] == provider: # remove same "provider:"
            link_name = link_split[1]

    cls.models_dir[provider][name] = link_name

delete_model classmethod

delete_model(res_name)

Delete a model entry.

Parameters:

Name	Type	Description	Default
res_name	str	Model entry in the form "provider:name".	required

Source code in sibila/models.py

@classmethod
def delete_model(cls,
                 res_name: str):
    """Delete a model entry.

    Args:
        res_name: Model entry in the form "provider:name".
    """

    cls._ensure()

    provider, name = provider_name_from_urn(res_name,
                                            allow_alias_provider=False)

    if provider not in cls.ALL_PROVIDER_NAMES:
        raise ValueError(f"Unknown provider '{provider}', must be one of: {cls.ALL_PROVIDER_NAMES}")

    prov = cls.models_dir[provider]        
    if name not in prov:
        raise ValueError(f"Model '{res_name}' not found")

    # verify if any entry links to name:
    def check_link_to(link_to_name: str, 
                      provider: str) -> Union[str, None]:

        for name,entry in cls.models_dir[provider].items():
            if isinstance(entry,str) and entry == link_to_name:
                return name
        return None

    offender = check_link_to(name, provider)
    if offender is not None:
        raise ValueError(f"Cannot delete '{res_name}', as entry '{provider}:{offender}' links to it")

    offender = check_link_to(name, "alias")
    if offender is not None:
        raise ValueError(f"Cannot delete '{res_name}', as entry 'alias:{offender}' links to it")

    del prov[name]

save_models classmethod

save_models(path=None, include_base=False)

Source code in sibila/models.py

@classmethod
def save_models(cls,
                path: Optional[str] = None,
                include_base: bool = False):

    cls._ensure()

    if path is None:
        if len(cls.models_search_path) != 1:
            raise ValueError("No path arg provided and multiple path in cls.search_path. Don't know where to save.")

        path = os.path.join(cls.models_search_path[0], "models.json")

    with open(path, "w", encoding="utf-8") as f:
        models_dir = cls.fused_models_dir() if include_base else cls.models_dir

        # clear providers with no models:
        for provider in cls.ALL_PROVIDER_NAMES:
            if provider in models_dir and not models_dir[provider]:
                del models_dir[provider]

        json.dump(models_dir, f, indent=4)

    return path

list_formats classmethod

list_formats(name_query, include_base, resolved_values)

List format entries matching query.

Parameters:

Name	Type	Description	Default
name_query	str	Case-insensitive substring to match format names. Empty string for all.	required
include_base	bool	Also list base_formats_dir.	required
resolved_values	bool	Return resolved entries or raw ones.	required

Returns:

Type	Description
dict	A dict where keys are format names and values are respective entries.

Source code in sibila/models.py

@classmethod
def list_formats(cls,
                 name_query: str,
                 include_base: bool,
                 resolved_values: bool) -> dict:
    """List format entries matching query.

    Args:
        name_query: Case-insensitive substring to match format names. Empty string for all.
        include_base: Also list base_formats_dir.
        resolved_values: Return resolved entries or raw ones.

    Returns:
        A dict where keys are format names and values are respective entries.
    """

    cls._ensure()

    out = {}

    name_query = name_query.lower()

    formats_dir = cls.fused_formats_dir() if include_base else cls.formats_dir

    for name in formats_dir.keys():

        if name_query and name_query not in name.lower():
            continue

        val = formats_dir[name]

        if resolved_values:
            res = cls.get_format_entry(name)
            if res is None:
                continue
            else:
                val = res[1]

        out[name] = val

    return out

get_format_entry classmethod

get_format_entry(name)

Get a resolved format entry by name, following links if required.

Parameters:

Name	Type	Description	Default
name	str	Format name.	required

Returns:

Type	Description
Union[tuple[str, dict], None]	Tuple of (resolved_name, format_entry).

Source code in sibila/models.py

@classmethod
def get_format_entry(cls,
                     name: str) -> Union[tuple[str,dict],None]:
    """Get a resolved format entry by name, following links if required.

    Args:
        name: Format name.

    Returns:
        Tuple of (resolved_name, format_entry).
    """

    cls._ensure()

    return get_format_entry(cls.fused_formats_dir(), name)

has_format_entry classmethod

has_format_entry(name)

Source code in sibila/models.py

@classmethod
def has_format_entry(cls,
                     name: str) -> bool:
    return cls.get_format_entry(name) is not None

get_format_template classmethod

get_format_template(name)

Get a format template by name, following links if required.

Parameters:

Name	Type	Description	Default
name	str	Format name.	required

Returns:

Type	Description
Union[str, None]	Resolved format template str.

Source code in sibila/models.py

@classmethod
def get_format_template(cls,
                        name: str) -> Union[str,None]:
    """Get a format template by name, following links if required.

    Args:
        name: Format name.

    Returns:
        Resolved format template str.
    """

    res = cls.get_format_entry(name)
    return None if res is None else res[1]["template"]

match_format_entry classmethod

match_format_entry(name)

Search the formats registry, based on model name or filename.

Parameters:

Name	Type	Description	Default
name	str	Name or filename of model.	required

Returns:

Type	Description
Union[tuple[str, dict], None]	Tuple (name, format_entry) where name is a resolved name. Or None if none found.

Source code in sibila/models.py

@classmethod
def match_format_entry(cls,
                       name: str) -> Union[tuple[str,dict],None]:
    """Search the formats registry, based on model name or filename.

    Args:
        name: Name or filename of model.

    Returns:
        Tuple (name, format_entry) where name is a resolved name. Or None if none found.
    """

    cls._ensure()

    return search_format(cls.fused_formats_dir(), name)

match_format_template classmethod

match_format_template(name)

Search the formats registry, based on model name or filename.

Parameters:

Name	Type	Description	Default
name	str	Name or filename of model.	required

Returns:

Type	Description
Union[str, None]	Format template or None if none found.

Source code in sibila/models.py

@classmethod
def match_format_template(cls,
                          name: str) -> Union[str,None]:
    """Search the formats registry, based on model name or filename.

    Args:
        name: Name or filename of model.

    Returns:
        Format template or None if none found.
    """

    res = cls.match_format_entry(name)

    return None if res is None else res[1]["template"]

set_format classmethod

set_format(name, template, match=None)

Add a format entry to the format directory.

Parameters:

Name	Type	Description	Default
name	str	Format entry name.	required
template	str	The Chat template format in Jinja2 format	required
match	Optional[str]	Regex that matches names/filenames that use this format. Default is None.	None

Source code in sibila/models.py

@classmethod
def set_format(cls,
               name: str,
               template: str,
               match: Optional[str] = None):
    """Add a format entry to the format directory.

    Args:
        name: Format entry name.
        template: The Chat template format in Jinja2 format
        match: Regex that matches names/filenames that use this format. Default is None.
    """

    cls._ensure()

    if "{{" not in template: # a link_name for the template
        if not cls.has_format_entry(template):
            raise ValueError(f"Could not find linked template entry '{template}'.")

    entry = {
        "template": template
    }
    if match is not None:
        entry["match"] = match

    cls.formats_dir[name] = entry        

set_format_link classmethod

set_format_link(name, link_name)

Add a format link entry to the format directory.

Parameters:

Name	Type	Description	Default
name	str	Format entry name.	required
link_name	str	Name of format that this entry links to.	required

Source code in sibila/models.py

@classmethod
def set_format_link(cls,
                    name: str,
                    link_name: str):
    """Add a format link entry to the format directory.

    Args:
        name: Format entry name.
        link_name: Name of format that this entry links to.
    """

    cls._ensure()

    if not cls.has_format_entry(link_name):
        raise ValueError(f"Could not find linked entry '{link_name}'.")

    cls.formats_dir[name] = link_name

delete_format classmethod

delete_format(name)

Delete a format entry.

Parameters:

Name	Type	Description	Default
name	str	Format entry name.	required

Source code in sibila/models.py

@classmethod
def delete_format(cls,
                  name: str):
    """Delete a format entry.

    Args:
        name: Format entry name.
    """

    cls._ensure()

    if name not in cls.formats_dir:
        raise ValueError(f"Format name '{name}' not found.")

    for check_name,entry in cls.formats_dir.items():
        if isinstance(entry,str) and entry == name:
            raise ValueError(f"Cannot delete '{name}', as entry '{check_name}' links to it")

    del cls.formats_dir[name]

save_formats classmethod

save_formats(path=None, include_base=False)

Source code in sibila/models.py

@classmethod
def save_formats(cls,
                 path: Optional[str] = None,
                 include_base: bool = False):

    cls._ensure()

    if path is None:
        if len(cls.models_search_path) != 1:
            raise ValueError("No path arg provided and multiple path in cls.search_path. Don't know where to save.")

        path = os.path.join(cls.models_search_path[0], "formats.json")

    with open(path, "w", encoding="utf-8") as f:
        formats_dir = cls.fused_formats_dir() if include_base else cls.formats_dir
        json.dump(formats_dir, f, indent=4)

    return path

info classmethod

info(include_base=True, verbose=False)

Return information about current setup.

Parameters:

Name	Type	Description	Default
verbose	bool	If False, formats directory values are abbreviated. Defaults to False.	False

Returns:

Type	Description
str	Textual information about the current setup.

Source code in sibila/models.py

@classmethod
def info(cls,
         include_base: bool = True,
         verbose: bool = False) -> str:
    """Return information about current setup.

    Args:
        verbose: If False, formats directory values are abbreviated. Defaults to False.

    Returns:
        Textual information about the current setup.
    """

    cls._ensure()

    out = ""

    out += f"Models search path: {cls.models_search_path}\n"

    models_dir = cls.fused_models_dir() if include_base else cls.models_dir
    out += f"Models directory:\n{pformat(models_dir, sort_dicts=False)}\n"

    out += f"Model Genconf:\n{cls.genconf}\n"

    formats_dir = cls.fused_formats_dir() if include_base else cls.formats_dir

    if not verbose:
        fordir = {}
        for key in formats_dir:
            fordir[key] = deepcopy(formats_dir[key])
            if isinstance(fordir[key], dict) and "template" in fordir[key]:
                fordir[key]["template"] = fordir[key]["template"][:14] + "..."
    else:
        fordir = formats_dir

    out += f"Formats directory:\n{pformat(fordir)}"

    return out

clear classmethod

clear()

Clear directories. Members base_models_dir and base_formats_dir and genconf are not cleared.

Source code in sibila/models.py

@classmethod
def clear(cls):
    """Clear directories. Members base_models_dir and base_formats_dir and genconf are not cleared."""
    cls.models_dir = None
    cls.models_search_path = []
    cls.formats_dir = None