Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Implements MapFields() #289

Open
wants to merge 4 commits into
base: develop
Choose a base branch
from

Conversation

Paulholio
Copy link

This is an attempt at eventually implementing the feature request from:

Issue 285: Updating All Fields

There is more work that needs to be done, but I think if we can spec out what types of operations we would want future users to be able to do across all fields, this can become very useful, especially on the validation and possibly security front. Let me know your opinions on this, please.

@hartym
Copy link
Member

hartym commented Sep 5, 2018

Hello,

Thanks for your contribution. A few things:

  • Let's call this MapFields, to be named like python's map() builtin (but refers to "fields", as we could have a Map that does this logic on a whole row)
  • Let's use the python callable to apply instead of a string : pass str.upper as parameter to the transformation, so it allows to use any callable the same way. The usage would then be MapFields(str.upper), or MapFields(lambda val: ...), etc. Much more flexible and less error prone as we'll need less code.
  • Please add an unit test (even if not very complete) to check that this continue to work over time, and to check that edge cases work as expected (what if we get an empty row? what if we pass mixed type rows?)
  • In the future, but let's not implement it in the first version, we could improve it by allowing the user to pass a list of fields to apply this to, or a callable to chose which fields to apply this to.

Thank you, this is a nice one !

@Paulholio
Copy link
Author

I have this renamed to MapFields and will have users pass in a function. Still need to write a unit test.

@hartym
Copy link
Member

hartym commented Sep 11, 2018

This pull request introduces 1 alert when merging 166e846 into ca464ef - view on LGTM.com

new alerts:

  • 1 for Except block handles 'BaseException'

Comment posted by LGTM.com

def _wrapfunc(func, val):
try:
retval = func(val) # Attempt to use the function
except:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should never use a "catchall" except, instead, catch Exception.

Not sure this behaviour is wanted though, as it may come to unpredictable behaviour.

nonlocal callFunc

b = {key: _wrapfunc(callFunc, value) for key, value in bag._asdict().items()} # iterate the wrapped callable
makenamedtuple = collections.namedtuple('Bag', ' '.join((b.keys()))) # constructor of the returned tuple
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use @use_context and get/set the input/output type from there. I think it's better than guessing the user would want this namedtuple.

@hartym hartym added the backlog label Jun 1, 2019
@hartym hartym changed the title Feature/update all fields number285 Feature: Implements MapFields() Jun 1, 2019
@hartym hartym added needs tests and removed backlog labels Jun 3, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

2 participants