Conversation
|
Let's finish this first and we'll focus on 1316 next. |
|
The code is currently broken, removing the annotation-level exclusions creates an issue with |
|
Ok I have completed the remaining tasks and fixed the tests. A couple of remarks:
|
|
Also added one commit to disable mapper-level configuration in favor of manager-level configuration only. I honestly do not see a valid use case for mapping the same class in two different ways under the same application, but I'm open to discussion. |
What about mapping two different classes with a different set of property exclusions? |
|
@olim7t cheers! sorry for not being available for a couple of days. everything looks great. let me know if there's anything else i can do regarding 1310. |
Hmm sorry but is this realistic? Even if it were, I don't think we have users willing to have this feature as of now, so why not add it later if need be? (it's not breaking) |
|
I added a commit for the inclusive/exclusive hierarchy bound. Two things bother me about transient properties:
I can't think of a good solution for 2. Maybe I'm overthinking this; if we're confident that shared exclusions are good enough, then we can keep the current implementation and I'm fine with a single, manager-wide configuration object. |
| public Builder withPropertyMappingStrategy(PropertyMappingStrategy propertyMappingStrategy) { | ||
| this.propertyMappingStrategy = propertyMappingStrategy; | ||
| public Builder withPropertyMappingStrategy(PropertyTransienceStrategy propertyTransienceStrategy) { | ||
| this.propertyTransienceStrategy = propertyTransienceStrategy; |
There was a problem hiding this comment.
I think withPropertyMappingStrategy needs to be renamed
0d3ec06 to
5f7f8a5
Compare
|
I am starting to have second thoughts about the usefulness of |
olim7t
left a comment
There was a problem hiding this comment.
(can only comment since I opened the PR)
Looks good to me with the latest changes to the strategies. PropertyAccessStrategy was the hardest to follow for me, but I think with a few changes and a bit of extra documentation it should be better.
|
|
||
| @Override | ||
| public boolean isTransient(String propertyName, Field field, Method getter, Method setter, Map<Class<? extends Annotation>, Annotation> annotations) { | ||
| return annotations.isEmpty() || annotations.containsKey(Transient.class); |
There was a problem hiding this comment.
From what I can tell, annotations is not filtered to the driver's types, so that would include a field that has any annotation except Transient.
| * @return a new {@link DefaultPropertyTransienceStrategy.Builder} instance. | ||
| */ | ||
| public static DefaultPropertyTransienceStrategy.Builder builder() { | ||
| return new DefaultPropertyTransienceStrategy.Builder(); |
There was a problem hiding this comment.
How about getting rid of the builder pattern for this class? It adds a lot of boilerplate, feels like if different behavior is needed it might take the form of alternate implementations.
| * | ||
| * @return {@code true} if the given field should be considered, {@code false} otherwise. | ||
| */ | ||
| boolean isFieldAccessAllowed(); |
There was a problem hiding this comment.
This (and isGetterSetterAccessAllowed) should be used in ReflectionUtils.scanFieldsAndProperties to completely skip scanning these members if they're not going to be used.
|
|
||
| /** | ||
| * Returns whether or not the given {@link Field} should | ||
| * be used when accessing its underlying property. |
There was a problem hiding this comment.
This threw me off at first read, generally when we say "given" it refers to a parameter. But this method has no parameters (maybe it used to?).
There was a problem hiding this comment.
Sorry the docs are stale and referring to a previous version of this class.
| import java.lang.reflect.Method; | ||
|
|
||
| /** | ||
| * A strategy to determine how mapped properties are read and written. |
There was a problem hiding this comment.
I think we should have a basic description of the scanning/mapping process detailing where each method is invoked, because it's hard to grasp from an outsider's perspective.
| * The map will include all annotations found on the field, the getter and the setter methods. | ||
| * @return {@code true} if the given property is transient (i.e., non-mapped), {@code false} otherwise. | ||
| */ | ||
| boolean isTransient(String propertyName, Field field, Method getter, Method setter, Map<Class<? extends Annotation>, Annotation> annotations); |
There was a problem hiding this comment.
I wonder if we should also include the mapped class here. It's not used right now but could prove useful. It cannot be derived from the field and getter/setter if those were inherited.
| /** | ||
| * Builder for {@link DefaultHierarchyScanStrategy} instances. | ||
| */ | ||
| public static class Builder { |
There was a problem hiding this comment.
When I added a builder here it was mainly for consistency with other configuration classes. Since I'm now advocating for removing the builders elsewhere, this one could go too.
| /** | ||
| * Builder for {@link MappingConfiguration} instances. | ||
| */ | ||
| public static class Builder { |
There was a problem hiding this comment.
Just for clarity, here it's justified to have a builder.
| Map<String, Object[]> fieldsAndProperties = new HashMap<String, Object[]>(); | ||
| Map<String, Field> fields = scanFields(baseClass); | ||
| List<Class<?>> classesToScan = Lists.<Class<?>>newArrayList(baseClass); | ||
| classesToScan.addAll(hierarchyScanStrategy.filterClassHierarchy(baseClass)); |
There was a problem hiding this comment.
Why not require that the strategy include the base class in the list? Then we could avoid that copy.
There was a problem hiding this comment.
Indeed, but wouldn't we have to check that the strategy did comply with this rule? It would make little sense to annotate a class with @Table and exclude it from mapping.
There was a problem hiding this comment.
If the contract says that it must be included, and an implementation doesn't, then it's a bug in the implementation and there's little we can do about it.
| <method>java.lang.String[] transientProperties()</method> | ||
| <justification>False positive, it's an annotation and the new method has a default value</justification> | ||
| </difference> | ||
|
|
There was a problem hiding this comment.
I forgot to remove this, it's not needed anymore.
| * <ol> | ||
| * <li>The property is annotated with {@link com.datastax.driver.mapping.annotations.Transient @Transient};</li> | ||
| * <li>The corresponding field is non-null and is marked with the keyword {@code transient};</li> | ||
| * <li>The property name has been explicitly black-listed.</li> |
There was a problem hiding this comment.
What does it mean for a property name to be explicitly blacklisted?
There was a problem hiding this comment.
Its name was passed to the constructor. This comment needs to explain it in more detail.
| FrozenValue.class | ||
| ); | ||
|
|
||
| private static final HashSet<String> DEFAULT_TRANSIENT_PROPERTIES = Sets.newHashSet( |
There was a problem hiding this comment.
wdyt about making this public? Although I don't expect it to be used very often, if one passes in transientProperties into the constructor they may be able to want to include these properties in addition to the ones they pass in.
| */ | ||
| public static class Builder { | ||
|
|
||
| private PropertyAccessStrategy propertyAccessStrategy = new DefaultPropertyAccessStrategy(); |
There was a problem hiding this comment.
This creates new default instances every time you construct a MappingConfiguration. This may be an overoptimization, but you could have a static singleton for each strategies default implementation so we don't need to construct defaults every time. On the other hand, creating MappingConfiguration should not be very frequent, so it might not be worth it.
There was a problem hiding this comment.
The problem is that I wanted to keep DefaultPropertyMapper really simple, so it's mutable, and not thread-safe. If we wanted to use a singleton pattern we would have to introduce a builder pattern as well and make the class immutable.
There was a problem hiding this comment.
I think it is fine the way it is then 👍
|
|
||
| /** | ||
| * Sets the {@link PropertyAccessStrategy property access strategy} to use. | ||
| * The default is {@link DefaultPropertyAccessStrategy}. |
There was a problem hiding this comment.
It might be nice to briefly summarize what the default strategy does for each of these, that way the user can decide if it is worth diving in to customizing their own strategy.
|
|
||
| } | ||
|
|
||
| @Test(groups = "unit", expectedExceptions = IllegalArgumentException.class, |
There was a problem hiding this comment.
Why are these tests no longer applicable? Shouldn't they still be invalid? Also noticed that the Invalid14 class still exists (but is no longer used)
There was a problem hiding this comment.
It looks like in the case of unreadable it throws an NPE (on a Preconditions check) and in the case of unwritable it logs:
[main] WARN com.datastax.driver.mapping.config.DefaultPropertyAccessStrategy - Property 'notWritable' is not writable and will not be mapped
but doesn't throw an exception and returns an EntityMapper without any columns.
There was a problem hiding this comment.
You are right, these exceptions have became log warnings, see here. Not sure though if we should throw an exception or just log a warning after all.
There was a problem hiding this comment.
I think throwing an Exception is probably a better solution as the user would likely want to be made aware that they are improperly mapping their entity.
There was a problem hiding this comment.
Makes sense I guess. I'll fix that.
| } | ||
|
|
||
| @Test(groups = "short") | ||
| public void should_inherit_annotations_up_to_highest_ancestor_exluded() { |
| @Test(groups = "short") | ||
| public void should_ignore_getters() { | ||
| MappingConfiguration conf = MappingConfiguration.builder() | ||
| .withPropertyAccessStrategy(new FieldOnlyPropertyAccessStrategy()) |
There was a problem hiding this comment.
Seeing this in use, it looks a lot less complicated. User only has to pass in the strategy they are interested in, and the provided implementations seem pretty viable and should cover a majority of use cases I would hope 👍
| } | ||
| } | ||
|
|
||
| public static class NoReflectionPropertyAccessStrategy implements PropertyAccessStrategy { |
| */ | ||
| public static class LowerSnakeCase extends CharDelimitedNamingConvention { | ||
|
|
||
| public LowerSnakeCase() { |
There was a problem hiding this comment.
Would be nice for there to be a static member for this instead of a custom class since there's only one way to configure it.
There was a problem hiding this comment.
Same for the others that only have default constructors too.
|
|
||
| } | ||
|
|
||
| abstract public static class CharDelimitedNamingConvention implements NamingConvention { |
There was a problem hiding this comment.
I don't think we should make this abstract since none of its members are abstract and might be useful to create anonymous naming conventions (although not sure why anyone would do that :D).
| throw new IllegalArgumentException(String.format("Property '%s': attribute 'value' of annotation @Computed is mandatory for computed properties", propertyName)); | ||
| return expression; | ||
| } | ||
| boolean caseSensitive = false; |
There was a problem hiding this comment.
One thing that surprised me was that if I defined a custom NamingStrategy, unless I have a @Column annotation on my field/method with caseSensitive=true, this automatically lowercases the property name. In the case where you are using a naming strategy where case is important (like most are), you would think that case sensitivity would be implied. What do you think about changing the logic such that if there is a NamingStrategy provided by the user that caseSensitive=true is implied? Otherwise you'll have to @Column(caseSensitive=true) everything which seems less than ideal.
There was a problem hiding this comment.
In addition, as you mentioned MapperCaseSensitivityTest.should_handle_case_sensitive_identifiers_when_keyspace_specified is failing. The reason is because The udt field name zipCode is being converted to zipcode because the default NamingStrategy enforces lowercasing of field name when getting the cassandra name representation.
I think it makes sense that if a user doesn't provide a NamingStrategy explicitly instead we could have a default strategy that does nothing to the input fields when mapping to cassandra column (and vice versa). We could then assume if a NamingStrategy is provided, we enforce case sensitivity, wdyt?
There was a problem hiding this comment.
Column.caseSensitive should only apply to Column.name if it's explicitly provided. And an explicit column name in the annotation should override the naming strategy for this property.
There was a problem hiding this comment.
I think initializing caseSensitive to true on this line would address the issue. If a naming strategy intends to ignore case, it should return lower-cased identifiers, and quoting them on line 378 has no effect.
Alternatively, we could change the flow to have two different return statements (name provided via annotation => consider caseSensitive, name provided via strategy => always quote), would make things a bit clearer I think.
There was a problem hiding this comment.
Pushed a commit to implement my suggestion.
|
The API seems really nice so far, will keep reviewing and will push some more tests tomorrow. |
|
can you rebase this? Would be good to do now since it's lagging a little bit behind 3.x. |
| */ | ||
| package com.datastax.driver.mapping; | ||
|
|
||
| public class PassThroughNamingStrategy implements NamingStrategy { |
There was a problem hiding this comment.
This is not used anywhere, do we need to provide it? Looks simple enough to reimplement if someone needs it.
| throw new IllegalArgumentException(String.format("Property '%s': attribute 'value' of annotation @Computed is mandatory for computed properties", propertyName)); | ||
| return expression; | ||
| } | ||
| boolean caseSensitive = false; |
There was a problem hiding this comment.
Pushed a commit to implement my suggestion.
|
|
||
| @PartitionKey | ||
| @Column(caseSensitive = true) | ||
| @Column(name = "userId", caseSensitive = true) |
There was a problem hiding this comment.
This is a behavioral change compared to the current version, but I think it's the reasonable thing to do. If someone uses case-sensitive Cassandra names with a strategy, they will likely have the strategy return mixed-case names rather than annotate all their fields.
There was a problem hiding this comment.
I like it! Also fixes my other complaint about having to annotate everything with @Column(caseSensitive=true).
There was a problem hiding this comment.
Yes this is a behavioral change but it solves the problem elegantly, so 👍
There was a problem hiding this comment.
Added note about that to upgrade guide in 3b86cef (hope that looks ok, feel free to adjust)
| /** | ||
| * Implementations of industry common naming conventions. | ||
| */ | ||
| public class CommonNamingConventions { |
There was a problem hiding this comment.
NamingConventions would be sufficient IMO
| * The following parts can be configured: | ||
| * <ul> | ||
| * <li>{@link PropertyMapper}: the mapper component that maps Java properties to table columns and UDT fields.</li> | ||
| * </ul> |
There was a problem hiding this comment.
This is a text description of the builder methods, I think the API is obvious enough that we can remove it, and avoid having to keep it in sync.
| CommonNamingConventions.LOWER_SNAKE_CASE)); | ||
| ``` | ||
|
|
||
| There is more to `DefaultPropertyMapper`; see the Javadocs and implementation for details. |
There was a problem hiding this comment.
I didn't want to cover every aspect, customizing the configuration is involved anyway, people who do it will likely look at the implementation.
|
Happy to see things kicking in...looking forward to using the new API. |
|
I think we are close to code-complete here. The only minor nit I would like to offer for discussion is: the builder pattern in |
|
rebased on 3.x |
|
A few test issues popped up after rebase, looking into it, probably an easy explanation. |
| mappedName = namingStrategy.toCassandraName(propertyName); | ||
| if (mappedName == null || mappedName.isEmpty()) | ||
| throw new IllegalArgumentException(String.format("Property '%s': could not infer mapped name", propertyName)); | ||
| return Metadata.quote(mappedName); |
There was a problem hiding this comment.
If the Field/Column doesn't have a name set, we depend on the field name, this explains the tests failing for UDT mapper tests(see this comment). I think we could just use handleId in MappedUDTCodec when looking up columnMappers and it should fix it, wdyt?
There was a problem hiding this comment.
Actually handleId does the opposite of what i want there (strips quotes).
There was a problem hiding this comment.
Maybe what we should be doing here is using Metadata.escapeId (note: this method is not accessible from this scope) instead of quote, which would only quote the name if it needs to be quoted. MappedUDTCodec would then work as is I believe. All the MapperUDTTest tests pass with that change.
There was a problem hiding this comment.
It looks like doing something like that will fix all the test issues since they seem to be tied to quoting.
There was a problem hiding this comment.
Yes, escapeId is the solution here. We can't always quote the name because it's compared against the schema metadata coming from Cassandra, where identifiers are only quoted if necessary.
I think we can make handleId public instead of duplicating the logic, it looks like something that might be useful for end users occasionally.
newkek
left a comment
There was a problem hiding this comment.
lgtm, had some comments. Mostly I was wondering if we could get rid of exposing the PropertyMapper class because I'm not sure if anybody would want to implement a different PropertyMapper considering we have the different mapping Strategies options. Imo the strategies offer great configuration already, so I'd be in favor of allowing to set the strategies via the MappingConfiguration directly, and not expose the PropertyMapper/DefaultPropertyMapper. The rest looks good, I also may have missed a part of the conversation but I was wondering if there is really a use for the javaConvention part of the NamingStrategy, do we really expect that many people to explicitly not follow the naming conventions of Java, in a Java program? There are probably use cases I am not aware of.
| public String toString() { | ||
| return propertyName; | ||
| } | ||
| public interface PropertyMapper { |
There was a problem hiding this comment.
I don't understand the need of an interface here, could there be another kind of implementation? Also I don't understand why we expose those 2 methods considering their default (and only) implementation will call the same method internally?
There was a problem hiding this comment.
could there be another kind of implementation?
There used to be a test that completely bypassed reflection in favor of hard-coded casting (in case the mapped classes are known in advance), but I can't find it anymore.
Also I don't understand why we expose those 2 methods considering their default (and only) implementation will call the same method internally
They call the same method but the second argument (the annotations allowed on the properties) is different.
There was a problem hiding this comment.
My intention was to distinguish two populations: "regular" and "advanced" users. Regular users would typically use DefaultPropertyMapper and just configure the strategies associated with it. Advanced ones would rather extend it or re-implement it completely.
The test that completely bypasses reflection has indeed been removed, not because it doesn't work anymore, but because it's imo something we should leave for the advanced users to do. But please trust me on this, I have tried, and it works (it's just a considerable amount of code to write).
There was a problem hiding this comment.
To be more specific: to bypass reflection you need at a bare minimum to 1) override DefaultPropertyMapper.createMappedProperty() and 2) provide your own implementation of MappedProperty.
There was a problem hiding this comment.
Fair enough, I am not convinced for the target of "advanced" users but we'll see.
| * @param transientPropertyNames a set of property names to exclude from mapping; may not be {@code null}. This | ||
| * will completely replace any names already configured for this object. | ||
| */ | ||
| public DefaultPropertyMapper setTransientPropertyNames(Set<String> transientPropertyNames) { |
There was a problem hiding this comment.
Is it possible now to specify a white list of properties to map? So that you can either: specify a list of properties to exclude, or specify a list of only the properties that will be included in the mapping (not considering the annotations).
I thought the original fix was allowing that but I just checked because I remember the WHITE_LIST enum but it was actually allowing to do the same thing than now, but what do you think of the idea?
There was a problem hiding this comment.
That feels less commonly needed, and as a last resort you can override isTransient.
I think what was called WHITE_LIST at some point was actually the equivalent of PropertyTransienceStrategy.OPT_IN.
There was a problem hiding this comment.
Seems it could be useful for when doing some sort of inheritance in the Java objects when having only one Cassandra table to map to. By providing a list of only the properties that are supposed to be mapped, you can map a table with a Java object, or its child, or its child's childs, etc, without having to explicitly exclude the children's properties via a black list each time. Or when you're denormalizing tables in C*, it could help selecting only some fields to map, without worrying about the other fields in the different denormalized tables, or other fields in the mapped object. Seems useful to me but that's a subjective opinion and we don't really have external feedback requesting this specifically so... ok.
| * achieve pre-<a href="https://datastax-oss.atlassian.net/browse/JAVA-541">JAVA-541</a> | ||
| * behavior. | ||
| */ | ||
| public class MappedClassesOnlyHierarchyScanStrategy implements HierarchyScanStrategy { |
There was a problem hiding this comment.
It seems like the purpose of this class is the same than if the user had used new DefaultHierarchyScanStrategy(TheMappedClass.class, true), is it ok to expose those 2 ways for doing the same thing?
There was a problem hiding this comment.
But TheMappedClass.class is not constant, filterClassHierarchy gets called for each class.
There are things that you can do by overriding protected methods that go beyond strategies (e.g. your comment about white list).
One example was if you use field access and your naming convention includes a prefix, for example |
Hungarian notation is still pretty much common, although I admit I've been seeing it less and less frequently. |
| <justification>False positive, the enclosing class is package-private so this was never exposed</justification> | ||
| </difference> | ||
|
|
||
| <difference> |
There was a problem hiding this comment.
I think this is ok? The method was protected, so in theory classes that extend BuiltStatement could access it, but you can't extend BuiltStatement directly since its abstract method buildQueryString is package private. I guess you could extend a subclass, but it isn't clear which one you could extend since they all seem to have package private constructors.
There was a problem hiding this comment.
I've re-added the method and deprecated it. It doesn't hurt to keep it and it's the cleanest/safest way.
|
|
||
| ```java | ||
| PropertyMapper propertyMapper = new DefaultPropertyMapper() | ||
| .setPropertyTransienceStrategy(PropertyTransienceStrategy.OPT_OUT); |
There was a problem hiding this comment.
Shouldn't this be OPT_IN instead of OPT_OUT based on the description above?
There was a problem hiding this comment.
Fixed (assuming i'm right since default is OPT_OUT)
| [@QueryParameters]: http://docs.datastax.com/en/drivers/java/3.0/com/datastax/driver/mapping/annotations/QueryParameters.html | ||
|
|
||
|
|
||
| ### Mapping configuration |
There was a problem hiding this comment.
This is probably the right amount of documentation 👍 . I think if the user wants to go beyond this they probably need to take time to understand the code. We can always add more based on questions we get from community.
|
Pushed up some more tests. I think at this point we are pretty good on coverage (can always add more to test more permutations but I think what we have is adequate enough). All of my comments/concerns have been addressed and things are working pretty well. 👍 'ing the PR, but will follow convo/commits as they come. |
|
I introduced a test failure (not using frozen on UDT)...fixing. |
To put it differently, attempting to change the configuration at runtime would have unpredictable consequences. |
6e232f6 to
3fd2e05
Compare
JAVA-1310: Make mapper's ignored properties configurable. JAVA-1316: Add strategy for resolving properties into CQL names. This is a squashed commit, including significant contributions from @adutra and @tolbertam.
|
I used the "squash and merge" strategy and I just realized it doesn't preserve the original commit author 😞 Sorry @avivcarmis I would have mentioned you in the commit message if I had known that. |

Reboot of @avivcarmis original pull request:
Reminder of the remaining tasks:
PropertyScanScopeclass will be changed to an enum containing{GETTERS_AND_SETTERS, FIELDS, BOTH}.HierarchyScanStrategyclass will be migrated to a single field of type Class<?> indicating the deepest ancestor to scan.=> I think it would also be useful to indicate if the bound is inclusive or not. In some cases you may have a common parent and you'll want to say "up to this parent included", but in others you may have multiple independent trees, and it will be "up to
Objectexcluded".Add this change.Document the fact thatMappingManagergetters will account for configuration object only for the first time, then they'll be ignored as discussed here.Rename transient config properties and document transient hierarchy (config transient props vs. class annotation transient properties vs. transient annotation) as discussed here.=> I've removed the annotation-level config propertyRenamePropertyMappingStrategyvalues to{OPT_IN, OPT_OUT}as discussed here.ReuseAnnotationParser.VALID_COLUMN_ANNOTATIONSas discussed here.Account for javatransientmodifier as discussed here (i'm actually +1 on that, fell for it myself when i started using the mapper).RenameMapperConfigurationtoMappingConfiguration.Last thing left open is whether we want to remove=> no, let's keep it separatePropertyScanConfigurationfield and move it's props toMappingConfigurationtop level.