Skip to content
Snippets Groups Projects

Fixed mario.wrap([dask]). It was not possible to set the npartitions kwarg

Closed Tiago de Freitas Pereira requested to merge fix-dask-wrap into master
1 unresolved thread
mario.wrap(["dask"],pipeline, npartitions=n)

wasn't working.

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
463 463 trans, leftover = _wrap(trans, **kwargs)
464 464 estimator.steps[idx] = (name, trans)
465 465
466 # Using the leftovers as new kwargs to be consumed further
467 kwargs = leftover
466 468 # if being wrapped with DaskWrapper, add ToDaskBag to the steps
467 469 if DaskWrapper in bases:
468 470 valid_params = ToDaskBag._get_param_names()
469 471 params = {k: kwargs.pop(k) for k in valid_params if k in kwargs}
470 472 dask_bag = ToDaskBag(**params)
471 473 estimator.steps.insert(0, ("ToDaskBag", dask_bag))
472
474 leftover = kwargs
  • mentioned in merge request !27 (merged)

  • !27 (merged) has already these modifications. closing this one

  • Please register or sign in to reply
    Loading