Ricardo's Blog

  • 03:35:25 pm on December 6, 2010 | 16
    Tags: , , ,

    While I’m a massive proponent of Maven, I’m sceptical of the benefits of nested modules. I think they encourage a lot of bad coding and build practices so, when I see people using them, I shudder. Here are my 4 reasons why:

    1. M2Eclipse does not support them: Not the best reason, which is why I’m listing it first (save the best for last). Given that a lot of people use Maven through M2Eclipse, I think it is a significant obstacle for day-to-day development. Actually, M2Eclipse used to support nested modules but removed support for various reasons.
    2. Brittleness: One of the nice features of Maven is the ability to loosely couple project dependencies. You don’t care where dependencies live, whether it’s in your Eclipse workspace, a local or remote repository, or some other kind of container altogether. But nested modules use sub-folders so everything has to sit in specific, relative folder locations. This creates a tight coupling between modules. Suddenly you have to care where your modules sit, making your project brittle.
    3. Time consuming: There is also a development overhead incurred by having to check out, test and build a bunch of other modules when you’re only making changes to a single module. Doesn’t sound like much but it all adds up, draining development time, consuming system resources. We’ve all sat there taping our fingers while our continuous integration server builds for a release.
    4. Encourages monolithic projects and discourages code re-use. Normally Maven encourages you to separate your components into individual projects. If your project needs to use a piece of code from another project, you simply re-factor that shared code into a new project and add it as a dependency to both. Once you’ve separated code into a new dependency it’s an independent blob, with it’s own life-cycle, and it’s clearly separated from other chunks of code. But with nested modules, different modules can all to easily cross-breed, becoming tightly coupled to the project code base and life-cycle, making it harder to re-use them in other projects.

    We all remember the problems with monolithic, Ant-based build scripts: brittle, difficult to integrate with other build tools, and slow. I don’t think nested modules are really much better. In fact, I would argue that using nested modules as described here is a Maven anti pattern.

    That said, I’m open to discussion, so if you have seen benefits with nested modules comment here and let me know.



  • Ruben 7:22 pm on December 6, 2010 | # | Reply

    Hi, I think nested modules, are really useful some of the time, if I am making a game both for android and pc using for example the libgdx library I don’t want to have to keep stuff in sync by hand. Those subprojects don’t exist alone, they are two parts of the same thing.

    About m2eclipse support, I think that what is not supported is using a single eclipse project to represent all the nested modules. If you checkout each module as a single project in eclipse it works perfectly, the dependencies between them are project dependencies in eclipse. It just works.

    • rgladwell 1:44 pm on December 7, 2010 | # | Reply

      @ruben I agree nested modules are a way around the problem of Maven’s “one artefact per POM”. You could have the main body of the code in the parent POM and a different artefact for the sub-modules. Although I’m still unsure of the benefits of this approach over having a separate, non-module project for each.

      thanks for the point on m2eclipse support, although you still have to have the nested modules in specific relative directories.

  • Tim O'Brien 1:06 pm on December 7, 2010 | # | Reply

    #1, This is an odd point, the support removed from m2eclipse was the ability to import multi-module projects as a single, aggregated project with multiple source locations. m2eclipse supports multi-module projects, please get your facts right.

    I’ll say it again so it doesn’t show up in someone else’s post: m2eclipse supports multi-module projects.

    #2, A parent expects sub-modules to be subdirectories. A child (not necessarily a sub-module) references a parent using a GAV coordinate. If you are just focused on that child relationship, there is no requirement to place a child in a subdirectory of a parent. Maybe you haven’t really understood the idea in practice?

    #3, huh? Again, this has no bearing on multi-module projects. Are you trying to do a release? Do you need to invest in a more capable continuous integration server?

    #4, nothing stops people from making bad decisions about coupling. This has nothing do do with multi-module projects, and everything to do with people making bad decisions.

    • rgladwell 1:49 pm on December 7, 2010 | # | Reply

      @tim wow, I almost unapproved your message for incivility. ruben made some of the same points, but very politely and without the completely unnecessary snark. be more polite.

      Although talking about getting facts right, my article talks about *nested modules* not *multi-modules*.

      #1 So in this case, m2eclipse certainly does not support nested modules as I described them in the article. The evidence is the URL sitting in the blog post right there!

      #2 yes, again, in *nested* module projects, sub-modules have to be in subdirectories. my point still stands.

      #3 sorry are you saying it doesn’t take longer? i’m pretty sure building everything does take longer than just building one component.

      #4 i’m pretty sure my point here was that *nested* modules *encourage* bad development. RTFA

      • Tim O'Brien 2:13 pm on December 7, 2010 | #

        Your post still confuses the idea of “nested modules” with “multi-module” projects.

        You are confusing the idea of “nested” modules with “multi-module” projects. I’ll add an additional note to the book to make it more clear.

        The point you are missing is that child – parent relationships are the important relationship. The sub-module relationship is really just there to aggregate builds. As builds grow larger and larger, you break up the build into separate multi-module projects.

        Maven can’t force people to make good decisions about modularity and project scope. (It takes a long time for my big project to build, has absolutely nothing to do with Maven or multi-module project structure.)

      • rgladwell 2:18 pm on December 7, 2010 | #

        I notice you haven’t actually explained how I’m confusing “nested modules” with “multi-module” projects. I think I’m quite clearly talking about the situation where you have checked out a parent module with several sub-modules.

        Maven doesn’t force anyone to do anything, it encourages them to make good decisions. Nothing I said contradicted that, so I still don’t understand your point here. My experience with nested modules is that they encourage developers to make very bad decisions, which is why I avoid them at all costs for the reasons in TFA.

    • Brian Reindel 3:36 am on February 15, 2012 | # | Reply

      Yes, you are correct, m2eclipse will import multiple modules in a file system hierarchy, but it’s worthless without support for an aggregated project structure. Why would I want 15 different projects cluttering my workspace when in essence, it is actually one single project. I hear the same excuse from m2eclipse supporters all the time. “Oh, it does support parent/child POM relationships.” No, no it doesn’t support it, not in the true sense of the word. If it did, I would be able to import it as a single project. However, I do consider this secondary to the biggest issue with m2eclipse, and that is the fact that it requires you to define “interesting” plugin execution cycles. That is code for, we no longer trust you to ignore plugin cycles we don’t understand, and require you to define them by writing connectors.

  • Brian Jackson 4:05 am on December 24, 2010 | # | Reply

    Ricardo, are you familiar with the advanced reactor options? I use them frequently while developing in multi-module projects to only rebuild the module(s) that changed. http://www.sonatype.com/books/mvnref-book/reference/ch06s02.html

    And in practice I’ve found multi-module projects have helped me avoid #4. Otherwise myself and my coworkers would need to create a brand new project in our CI and release system for every single component of a “project”. Multi-module projects let me organize one overarching project into independent components, and a few dependent modules to generate aggregate artifacts like WARs and EARs, while still maintaining one versioning scheme for building and releasing. So in practice my team only needs to press one button to release a project rather than needing to know the relationship between modules and the order they need to be released. My largest project right now has 38 modules so it would be prohibitive to manage them as separate projects, but at the same time it’s structured in a way that is far from monolithic.

    • rgladwell 12:52 pm on December 30, 2010 | # | Reply

      Hi Brian

      Thanks for your thoughtful response. I wasn’t aware of the advanced reactor options, and it’s certainly a nice feature that you can selectively build modules.

      I would argue there already exists a mechanism for separating logical parts of code in Java: packages. If
      your code is being built as one artefact, then I don’t see the benefit of using nested modules over a good packaging strategy.

      On #4 I suppose what I meant to say is that there is a benefit from requiring your project components build separately. It creates a clear distinction between your code, encouraging decoupling. It also makes any build issue like cyclical dependencies quickly obvious. Nested
      modules remove these benefits, encouraging tighter coupling.

      • Brian Jackson 8:38 pm on December 30, 2010 | #

        In my case, there are three benefits to multi-module projects, code organization at a higher level than packages (which you alluded to but seem to have dismissed), build/release management (which I mentioned earlier) and dependency simplification (a biggie as I’ll explain below).

        My service is not being built as one artifact, so that seems like where we diverge. I have one service that is comprised of several EARs (6 to be exact). Unfortunately the term ‘service’ is a bit ambiguous, what I’m referring to a logical combination of deployables that make up one production application. The additional modules separate the WARs, EJB JARs, and dependent POJO code (in JARs), and then a set of modules that capture any independent code shared across the 6 end-product EARs, grouped by their dependencies (ex. All classes that need JPA in one, all classes that need JMS in another).

        Also, one of the shared modules groups the set of classes that internal Java clients of our service need into one JAR that has no additional dependencies. This keeps other teams/apps/services from needlessly incorporating dependencies. I’m sure that’s not a new idea to you, but it sounds like you have gone the route of extracting a module like this into its own Maven project and managing its builds and releases independently. With almost 40 modules that need to be released together, and server/client version numbers need to match, for release management simplicity and developer sanity organizing it as one multi-module project that all gets cut as a version together makes a lot of sense for me.

        It is also a “separation of concerns” problem when dealing with a many-team corporate environment from:
        ** engineers working on the project: “keep it lowly-coupled/highly-cohesive within the project, but don’t make me trace pom files to figure out where all the internal dependencies are coming from!”,
        ** to operational/system engineers: “treat it as one product and don’t confuse me with the implementation details, hand me 38 jars and I’ll kill you”,
        **to engineers as ‘users’: “don’t make me depend on more than I need in my service just because I’m your client, for instance JPA, Hibernate, SQL Server JDBC drivers, etc just because that is how YOU deployed it but I just needed some of your POJOs”.

        Then when you throw in QA, DB Ops, Dev Ops, and every other team in a corporate environment, and there are a lot of “concerns” to be concerned about. Not that every project where I work is like that, but its definitely something that is easier to plan for upfront. But for me it didn’t feel like ‘planning’ when multi-modules projects become a ‘natural’ way of organizing an overarching “service”.

        And to your last point, about multi-module projects allowing cyclic dependencies and encouraging coupling, I just don’t see your perspective yet since I have the exact opposite view of multi-module projects, they enforce non-cyclic Maven dependencies the same as any artifact and do nothing more to encourage coupling than any bad programmer could do without using multi-module projects. If you wouldn’t mind expanding with a more concrete example maybe, and then I’ll have a clearer idea of what you mean.

      • rgladwell 6:53 pm on January 2, 2011 | #

        Surely the only meaningful “code organization at a higher level than packages” is a JAR file? That is ignoring archive formats that aggregate other JAR and class files such as WARs.

        I always assumed that you should only create separate JAR files if the code is going to be re-used. If it’s only ever deployed with one specific application, you shouldn’t need a higher level than packages. OTOH if your code is re-used by more than one project, then it will inevitably need an independent life-cycle in which case a separate, independent project is probably desirable.

        Sorry, but I’m not sure I entirely follow your example, or fully understand why you need 6 EAR files for one application. Do the EAR files do different things? What do you gain by not combining them into a single EAR file, if they are only ever deployed together as a single unit?

        Also I’m not sure I understand what is so bad about making software engineers “figure out where all the internal dependencies are coming from!”. Tools like m2eclipse make this sort of dependency management trivial.

        And on the subject of “engineers as ‘users'” who ask that you “don’t make me depend on more than I need”, this indicates that your code is being re-used in other projects. In which case I can’t see what you gain using nested modules versus separate projects. Perhaps your common libraries are too course and should be more finely grained, with careful pruning of your dependency tree.

        I also note that many of the parties you list (QA, DB Ops, operational/system engineers) shouldn’t care about whether you use nested modules or not. Sorry, maybe I’m not following you.

        P.S. Identation is making this thread un-readable, please feel free to continue this discussion via email. Please click on my gravatar icon for contact details.

  • Ted 7:22 pm on May 31, 2011 | # | Reply

    All of this is well and good until you run into memory issues when your number of maven projects gets up into the mid double digits (our current project requires 8GB to compile). It’s also difficult to navigate through the structure when every artifact is a top level element. Scrolling through projects instead of collapsing larger containers is much more difficult. Our dilemma now is do we wait for eclipse to have native maven support or switch IDEs.

    • rgladwell 9:51 am on July 15, 2011 | # | Reply

      If your maven projects are in the double digits I imagine you have an architectural problem no build system can help you with.

      • Anonymous 9:29 pm on March 13, 2012 | #

        Not necessarily true. If you have system of applications sharing code base then this becomes part of your life. Just to prevent question” so why don’t you create single application” I will answer: because you have distinct requirements for every application and they rarely overlap… including region of the world, support, interactions, users, business e.t.c. We are talking scale of the problem that cannot be sorted by common architecture across all.

      • rgladwell 10:53 am on March 14, 2012 | #

        I’ve worked on several large projects and I’ve never seen any one project have more than 10 internal projects. I’m all for code re-use, but if you have double-digit internal projects as part of your build you have a severe case of NIH syndrome.

  • Albert Liptay 9:00 pm on January 10, 2012 | # | Reply

    omg, I am just so frustrated with multi modules, I’ve tried, tried again to understand the rational when to use them and when not to use them. But seriously, I’m ripping apart yet another multi-module because I just can’t get it to work. Maven is all about convention over configuration, and I’m sorry multi-modules are not simplifying my life by convention.

    PS: Using SVN and Eclipse which doesn’t help the situation at all..

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: