Hitachi Vantara Pentaho Community Wiki
Child pages
  • 01. Open Source - Advantages
Skip to end of metadata
Go to start of metadata

The principles listed in the previous page are such that in combination they attract a variety of participants to open source projects. The participants contribute in many ways, some of which are listed below. In reviewing these types of contributions I will note what effect each one has on the software or the project. One of the main effects of community participation is that the software gets 'hardened'. By hardened I mean that the stability, usability, performance, security and fitness for use of the software has been improved by the actions of the community. Hardening is mentioned a lot from this point forward not because it is a central principle of open source but because it is important from a methodology perspective: it determines when the software is 'ready'.

Peer Review

By providing peer review of the feature roadmap, architecture, and design documents the participants challenge and validate the requirements and design of the software. This helps to reduce the numbers of specification defects and design defects. Finding these defects early is important because the implementation cannot be complete, and therefore implementation defects cannot be identified, until the specification and design are stable.

  • Positive Peer Review: Indicates that the specification and design are acceptable. This kind of feedback is rare.
  • Negative Peer Review: Indicates that the specification and design is not likely to support the use cases of the community. This feedback is used to improve the requirements and design of the software.
  • No Reviews: Indicates a lack of participation or a tacit approval. These two are hard to distinguish between.

Use Cases

Many participants come to an open source project with a use case that is different in some way to everyone else's. In attempting to use the software to solve their problem each participant is likely to exercise the software in different ways. A large community of participants with unique use cases provide a very effective test-bed to identify specification and design defects. If the software has been accepted for sufficiently numerous and diverse use cases it indicates that the specification and design are acceptable.

  • Positive Use Case Feedback: Indicates that the software supports the community's many use cases. This kind of feedback is rare.
  • Use Case Problems: Indicates that the software does not support the use cases of the community. This feed-back is used to improve the requirements of the software.
  • No Evidence of Use Cases: Indicates a lack of participation or a tacit approval. These two are hard to distinguish between.

Finding Bugs

In many cases it is harder to find bug than it is to fix them. Once specification and design defects have been removed the community can uncover and report bugs in the implementation. This helps reduce the number of implementation defects.

  • Reports of High Quality: Indicates that the software has few defects. This kind of feedback is rare.
  • Lots of Bug Found: Indicates that the software has defects. This feedback is used to improve the quality of the software.
  • Few Reports: Indicates a lack of participation or high quality. Without visibility into the use cases and usage patterns of the community this is very hard to measure.

Fixing Bugs

In most cases the participant who finds a bug is not the one who fixes it. These contributions reduce the number of implementation defects. In many open source projects the majority of the bug fixes are made by the core developers of the project.

  • Bugs Fixed by Community: Indicates that the community is participating at the code level and that they have enough knowledge of the source code to do this. In many open source projects the core developers do a very high percentage of the coding.
  • No Bugs Fixed by Community: As mentioned before finding the bugs is harder than fixing them and a low level of contribution is not necessarily a problem.

Usability

New community members, unfamiliar with the software, are frequently the ones who identify aspects of the software that are obscure or hard to use.

  • Positive Feedback: Indicates that the software is easy to understand and use. This kind of feedback is rare.
  • Negative Feedback: Indicates that the software is not easy to use. This feedback usually manifests as forum posts asking how to install the software or use a feature. This feedback is used to improve the usability of the software.
  • Absent Feedback: Indicates a lack of participation or high levels of usability. Without visibility into the use cases and usage patterns of the community this is very hard to measure.

Documentation Updates

New community members, unfamiliar with the software, are frequently the ones who find omissions, inaccuracies, or staleness in the documentation. Since documentation is part of the 'deliverable' these contributions help find and remove defects

  • Reports that Documentation is Good: Indicates that the documentation is well structured and complete. This kind of feedback is rare.
  • Documentation Issues Reported: Indicates that the documentation is badly structured or not complete. This feedback usually manifests as forum posts asking how to install the software or use a feature or pointing out a step in the documentation that did not work. This feedback is used to improve the accuracy and completeness of the documentation.
  • No Documentation Issues Reported: Indicates a lack of participation or good documentation. Without visi-bility into the use cases and usage patterns of the community this is very hard to measure.

Platform Testing

Along with unique use cases the participants have a wide variety of hardware, operating systems, environments, and touch points with applications that the software will be applied to. Knowing that the software is executing successfully in these diverse environments indicates that it is robust.

  • Positive Feedback: If no platform-related issues are reported by the community and it is known that the platform coverage is high it indicates that stability is. This kind of feedback is rare.
  • Negative Feedback: Indicates that the software has problems on different hardware or operating system environments. This feedback usually manifests as forum posts reporting problems. It is important to know the environment of the participant otherwise this might not be recognized as a platform issue. This feedback is used to improve the stability of the software in different environments.
  • Absent Feedback: Indicates a lack of participation, or high stability on lots of platforms, or a community with little platform diversity. Without visibility into platforms that the community are using this is very hard to measure.

Scalability and Performance

Participants will often compare the scalability and performance of an open source project with an alternative open source or proprietary offering and provide detailed information on bottlenecks and limiting factors.

  • Positive Feedback: Performance and scalability are good. This kind of feedback is rare.
  • Negative Feedback: Indicates that the software has performance issues. This feedback is used to improve the scalability and performance of the software.
  • No Feedback: Indicates a lack of participation, or good. Without visibility into the uses cases of the community this is very hard to measure.

New Features

The community will also contribute new features. If an open source project provides most, but not all, of the features that a participant needs it might be in their best interest to add the feature to the software and contribute it to the project (so that they don't have to perform ongoing maintenance and enhancement of the feature).

  • New Feature Contributions: Indicates that an active community is participating in the evolution of the software.
  • No Feature Contributions: Indicates that there is not an active community, or that the existing features meet the needs of the community. Without positive acceptance feedback from the community this is very hard to measure.

Forum Help

When new members of the community ask questions or report problems on the project forums more experienced community members (not the core developers) often answer the posts. This helps the core developers concentrate on the source code. If community members are helping each other out on the forums it is evidence of an active community. When a project is initially created the administrators and core developers answer most if not all the forum posts. As the project grows the more experienced community members become capable of helping the newer members.

  • No forum posts: Indicates there is either no community or that the software and documentation is perfect and no-one has any comments or questions (pretty unlikely).
  • Questions from new-comers and answers from the core team: Indicates a growing community that has yet to develop an extended team that can help answer questions.
  • Questions from new-comers and answers from the extended team: Indicates an active and knowledgeable community.

Localization

This helps to increase the volume of other contributions by expanding its potential community.

  • No Localization Contributions: Indicates there is either no community, that the existing localization support is sufficient, that the software is not interesting to some locales, or that the usage of the software is not easily understood by some locales.
  • Localization Contributions: Indicates that there is a geographically diverse community.
    Comparisons, Recommendations, and References
    This helps to increase the volume of other contributions by directing potential community members to the project.
  • No Recommendations or References: Indicates there is either no community, or that the community is not supportive or vocal, or that the project administrators have not asked for this kind of feedback.
  • Recommendations or References: Indicates that the community is supportive and vocal.

Contribution Classifications

For the purposes of this paper the contributions above can be put into two groups depending on whether the contribution directly helps to harden the software or not.

HARDENING CONTRIBUTIONS

NON-HARDENING CONTRIBUTIONS

Peer Review

New Features

Use Cases

Forum Help

Finding Bugs

Localization

Fixing Bugs

Comparisons, Recommendations, and References

Usability

 

Documentation Updates

\

Platform Coverage

 

Scalability and Performance

 

It is logical that the amount and tone of these contributions can be used to assess the extent to which the community has hardened the software. As noted for many of the contributions above the feedback and actions of the community can be interpreted in two opposing ways and can only be done effectively if the usage of the software, in terms of volume and diversity, is known.

To give it an analogy hardening is like putting the software on an Anvil of Truth and letting the community bang on it without rules or restrictions.

Contribution Factors

Along with the kinds of contributions listed above there are a few factors that affect the volume and quality of the contributions and hence the extent to which the software get hardened.

  • Use cases: How unique are they? This affects the volume and the overall quality of the contribution.
  • Usage: Is the software being evaluated, used in development of something, used in a testing environment, or being used in a production (live) environment? This will affect the kind of contribution that is likely to result. This affects the diversity of the contributions.
  • Motivation: Does the participant have a need to deliver something under a deadline or are they participating more out of interest? This will affect the likelihood that a participant will contribute anything in a given period.
  • Window of Opportunity: How long are the administrators or core developers of the software giving the community the opportunity to contribute? This will affect the volume of contributions.
  • Size of community: How large is the community of contributors? This will affect the volume of contributions.
    Due to the individual nature of each participants motivation and time-lines contributions are not offered to an open source project according to cleanly defined phases. Members of the community could be find bugs, offering features, and reviewing requirements documents on any given day regardless of where the current release of the software is in its process. This means that open source projects need to have a flexible approach to change management. A waterfall methodology does make sense for an open source project unless contributions and participation are not wanted.
  • No labels