Skip to content
Snippets Groups Projects

Local base build

Local base build
local-base-build into master
All threads resolved!
Merged Amir MOHAMMADI requested to merge local-base-build into master
All threads resolved!

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Amir MOHAMMADI
    Amir MOHAMMADI @amohammadi
    Author Owner

    @andre.anjos this patch fails on bob.conda when running bdt local base-build -vv nopython.txt could you test it to see if it works on your setup. Mine fails on vlfeat with:

    ================================================================================
Building "conda/kaldi" for python-3 (14/42)
================================================================================
...
...
================================================================================
Building "conda/vlfeat" for python-. (15/42)
================================================================================

INFO:bob.devtools.scripts.ci@2019-08-07 10:55:12,551: Conda build configuration file: /home/amir/idiap/git/bobs/bob.conda/conda_build_config.yaml
INFO bob.devtools.scripts.ci:<lambda>(73): Conda build configuration file: /home/amir/idiap/git/bobs/bob.conda/conda_build_config.yaml
INFO:bob.devtools.build@2019-08-07 10:55:12,552: Using the following channels during (potential) build:
  - http://www.idiap.ch/software/bob/conda
  - defaults
INFO bob.devtools.build:base_build(519): Using the following channels during (potential) build:
  - http://www.idiap.ch/software/bob/conda
  - defaults
INFO:bob.devtools.build@2019-08-07 10:55:12,552: Merging conda configuration files...
INFO bob.devtools.build:base_build(522): Merging conda configuration files...
Traceback (most recent call last):
  File "/home/amir/miniconda/bin/bdt", line 11, in <module>
    load_entry_point('bob.devtools', 'console_scripts', 'bdt')()
  File "/home/amir/miniconda/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/amir/miniconda/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/amir/miniconda/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/amir/miniconda/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/amir/miniconda/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/amir/miniconda/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/amir/idiap/git/bobs/bob.devtools/bob/devtools/scripts/bdt.py", line 42, in _decorator
    value = view_func(*args, **kwargs)
  File "/home/amir/miniconda/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/amir/idiap/git/bobs/bob.devtools/bob/devtools/scripts/local.py", line 216, in base_build
    ctx.invoke(ci.base_build, order=order, dry_run=dry_run, group=group, python=python)
  File "/home/amir/miniconda/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/amir/idiap/git/bobs/bob.devtools/bob/devtools/scripts/bdt.py", line 42, in _decorator
    value = view_func(*args, **kwargs)
  File "/home/amir/idiap/git/bobs/bob.devtools/bob/devtools/scripts/ci.py", line 348, in base_build
    condarc_options=condarc_options,
  File "/home/amir/idiap/git/bobs/bob.devtools/bob/devtools/build.py", line 530, in base_build
    metadata = get_rendered_metadata(recipe_dir, conda_config)
  File "/home/amir/idiap/git/bobs/bob.devtools/bob/devtools/build.py", line 168, in get_rendered_metadata
    return conda_build.api.render(recipe_dir, config=config)
  File "/home/amir/miniconda/lib/python3.6/site-packages/conda_build/api.py", line 44, in render
    permit_unsatisfiable_variants=permit_unsatisfiable_variants)
  File "/home/amir/miniconda/lib/python3.6/site-packages/conda_build/render.py", line 745, in render_recipe
    m = MetaData(recipe_dir, config=config)
  File "/home/amir/miniconda/lib/python3.6/site-packages/conda_build/metadata.py", line 853, in __init__
    self.parse_again(permit_undefined_jinja=True, allow_no_other_outputs=True)
  File "/home/amir/miniconda/lib/python3.6/site-packages/conda_build/metadata.py", line 924, in parse_again
    bypass_env_check=bypass_env_check),
  File "/home/amir/miniconda/lib/python3.6/site-packages/conda_build/metadata.py", line 1474, in _get_contents
    env.globals.update(ns_cfg(self.config))
  File "/home/amir/miniconda/lib/python3.6/site-packages/conda_build/metadata.py", line 94, in ns_cfg
    py = int("".join(py.split('.')[:2]))
ValueError: invalid literal for int() with base 10: ''
    Edited by Amir MOHAMMADI
  • André Anjos
    André Anjos @aanjos
    Owner

    The nopython.txt builds should not set the python version. Something is fishy there.

  • Amir MOHAMMADI
  • Amir MOHAMMADI
    Amir MOHAMMADI @amohammadi
    Author Owner

    @andre.anjos something doesn't make sense here. Why do we have nopython.txt and python.txt? conda build can handle building for multiple python versions.

  • André Anjos
    André Anjos @aanjos
    Owner

    I'm happy to merge whatever you have in mind, if that works, naturally, and improves compatibility.

  • André Anjos
    André Anjos @aanjos
    Owner

    python and nopython - if my memory serves me well, we would like to trigger the build of packages and set the python version to which the packages must build with.

    This allows us to enable/disable the builds of various python versions from the CI scripts themselves. It was like this at bob.admin and I just copied that behaviour here.

  • André Anjos
    André Anjos @aanjos
    Owner

    If there is another way to trigger this, then please describe how this trigger would work and, then, implement it as such (please ensure to change all required CI support files in this case).

  • André Anjos
    André Anjos @aanjos
    Owner

    Maybe you also set this as "WIP"?

  • Amir MOHAMMADI
    Amir MOHAMMADI @amohammadi
    Author Owner

    It was like this in bob packages but not in bob.conda https://gitlab.idiap.ch/bob/bob.conda/blob/33d61ad0b16d8c0456539835360b3ef5fd344a4d/.gitlab-ci.yml

  • André Anjos
    André Anjos @aanjos
    Owner

    So, a copy-n-paste issue?

  • Amir MOHAMMADI
    Amir MOHAMMADI @amohammadi
    Author Owner

    yes it went away after the move to bdt. It was being built with this script before: https://gitlab.idiap.ch/bob/bob.conda/blob/33d61ad0b16d8c0456539835360b3ef5fd344a4d/scripts/build.sh

  • Amir MOHAMMADI
    Amir MOHAMMADI @amohammadi
    Author Owner

    Also I don't get why you have a mini-channel here in bob.devtools. I think your main problem could have been the face that we need multiple conda_build_config.yaml files.

  • Amir MOHAMMADI added 1 commit

    added 1 commit

    • 6f34b90a - Add a local base-build command

    Compare with previous version

  • Amir MOHAMMADI resolved all discussions

    resolved all discussions

  • Amir MOHAMMADI
    Amir MOHAMMADI @amohammadi
    Author Owner

    Fixed this @andre.anjos you may merge it.

  • André Anjos
    André Anjos @aanjos
    Owner

    OK

  • André Anjos enabled an automatic merge when the pipeline for 6f34b90a succeeds

    enabled an automatic merge when the pipeline for 6f34b90a succeeds

  • André Anjos merged

    merged

  • André Anjos mentioned in commit a49f5b7d

    mentioned in commit a49f5b7d

    • Please register or sign in to reply