Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)

[Kapil Panchal <ka...@gmail.com>]

Hi Adam,

I’m currently working on *FINERACT-2290* and have a few questions before I
submit a pull request.

The *Collection Sheet API* in its current state appears non-functional and
contains several logical errors. It seems there was an earlier attempt to
convert from a JSON string request parameter to a class-based request
object, but:

Certain fields are missing.

The serializer is not correctly populating the objects, which causes the
conditional checks to be bypassed and results in incorrect (false)
responses.

This change set is *high risk* because it touches most of the loan and
savings product logic. I’ve had to refactor almost all major methods.
Extensive integration and end-to-end testing will be required to ensure
there are no regressions, especially in edge cases. At present, there are
no unit or integration tests for this functionality, and test creation is
outside the current ticket scope. I’ve been iterating on this for a while,
and only today have I reached a stable state after several experimental and
build-breaking attempts.

*Key Observations:*

Service methods are not annotated with @Service; instead, beans are defined
manually.

Repository wrappers are annotated with @Service. This mandates full unit
test coverage for these methods, but they should ideally be annotated with
@Component.

I agree with prior discussions on separating bean validation — having a
dedicated @Component validation class allows the request object to handle
checks independent of database queries.

Validation components can also perform database-related validations; these
can be injected into service classes for cleaner architecture.

Such validation components should be placed in *Fineract-Core* so they are
reusable across modules, reducing future refactoring needs.

The current design of having commands in *Fineract-Core* and
handlers/services/repositories in respective modules is good — it cleanly
decouples command definition from execution.

There is extensive use of this. in singleton contexts (API, Service,
Repository). While not harmful, it’s unnecessary boilerplate.

Multiple redundant intermediate DTOs exist where the request DTO itself
could be reused for data transfer.

I found redundant logic — e.g., a for loop with a break statement that
effectively executes only once; this can be simplified.

Some JDBC template queries use reserved SQL keywords, causing exceptions.
Refactoring these queries resolves the issue and returns proper response
objects.

*Suggestions:*

*Where appropriate, large tickets should be broken into subtasks to manage
complexity and reviewability.*

It may help to have a dedicated *developer-only Slack channel *for
technical discussions. This could complement other community spaces if
there’s a need to keep certain conversations more focused.

What are your thoughts on the above?

Thanks,
Kapil

Re: Mission of Fineract discussion (was Re: Collection Sheet Deprecation ...)

[James Dailey <ja...@gmail.com>]

I’d like to highlight this discussion of a year ago and suggest we
resurface any debates, and to get updates on vendors continuing to support
the project with contributions of code.

On Mon, Sep 1, 2025 at 7:40 AM Ed Cable <ed...@mifos.org> wrote:

> Thank you everyone for this valuable discussion and the inputs thus far.
>
> To assuage the concerns of Zayyad, Sifiso, Felix and others, Mifos,
> regardless of what decision the Fineract community makes in regards to
> deprecation of collection sheet functionality, Mifos will not be removing
> this functionality nor diminishing its focus on microfinance and financial
> inclusion within the Mifos X distribution nor within the Mifos X community.
> This is at both the back-end as well as front-end UI level, and our goal is
> to make these features more configurable via the UI so they can be
> supported from the back-end but not cluttering the user experience for
> those who don't need it. I'll make sure those who are working on
> maintaining these key pieces of functionality are sharing regular updates
> via community channels.
>
> I agree with Paul's articulation of the broader umbrella and bigger tent
> that Fineract helps to provide as well as the need for various
> individuals in the community representing certain verticals of financial
> services using the software to be more participatory in the upstream
> communities (ideally this would be in the form of code contribution, but at
> minimum it could be staying on the latest versions of the codebase, and
> providing active and engaged feedback in terms of new requirements, bug
> reports, usability feedback, etc.) I know that over the years Mifos has
> struggled to get proper user feedback for intern projects for
> requirements/features that do relate to microfinance.
>
> *So I think one key point is if the community wants its voice heard and
> the respective project to meet their ongoing requirements, they need to be
> engaged, active, and participatory* - I recognize that the mailing list
> might not be that best place for many of the users so that's why Mifos has
> an active Slack channel, has experimented with other communication
> tools/formats, and hosts regular community calls, and engages one on one
> with vendors representing these verticals.
>
> Concerning the focus on new features and functionality for microfinance
> and financial inclusion, part of the reason why there has been so little
> new development here has been the silver lining of this feature set being
> mature and near fully-functional. As history recounts, JLG lending was the
> original use case for Mifos, expanding to a broader feature set beyond
> lending for cooperatives and SACCOs and more of the organic contribution
> that has occurred with more advanced lending and formal financial services.
> That being said, Mifos is working with members of the ecosystem and
> community to enhance teller operations, better support savings-led
> approaches like the VSLA methodology, and better Shariah-compliant products
> for microfinance to name a few.
>
> I don't think we should view the sponsored contributions of more advanced
> lending features that Felix has highlighted as dominating the roadmap nor
> diminishing the platform's relevance and impact for microfinance and
> financial inclusion. Mifos has been shepherding this commercial project
> yielding these contributions over the past several years and we've been
> very deliberate about making the functionality as generic as possible and
> much of the functionality does add benefit to all users of the software.
> However these enhancements and how they can be configured could be better
> documented which is underway. Beyond the upstream code and the strong
> reference this customer provides, the Pepper Soup project has had many
> other benefits for the sustainability of the organizations stewarding the
> community to ensure that the focus can remain on the mission, timely
> releases are made, etc.
>
> A main reason why these contributions seem to overshadow others goes back
> to the point Paul raises is that others aren't contributing and the
> community needs to step up and contribute and participate upstream. This
> customer made the conscious decision and commitment to contribute all its
> enhancements upstream. If others in the community were willing to do the
> same with the features they've been building in their downstream solutions,
> we'd see a more balanced flow of contribution upstream. There are no
> barriers for these contributions coming in as long as they follow the
> community process. Mifos recognizes this requires a bit of handholding and
> we continue to try to enable more individuals and vendors to contribute
> upstream.
>
> Felix does raise a good point around the process and visibility into a
> roadmap and that is something James has been more proactive about with
> Fineract and Mifos will be doing as well in regards to Mifos X to provide
> the forward-looking view of the software, gather inputs on where others
> would like to contribute, highlight gaps that need closing, and signal to
> the community where help is needed in contributing to close these gaps.
>
> A last point I'll make is recognizing that if we are successful in both
> maintaining the existing feature set and merging in more upstream
> contributions to support the wide number of verticals that Mifos and
> Fineract attracts, the user experience of the software does get complicated
> and overwhelming and the software can be hard to maintain. Mifos is
> committed to ongoing modularity at both the front and back-end and is
> advancing efforts led by Aleks Vidakovic to achieve the same.
>
> So rest assured that Mifos will ensure the feature set for microfinance
> and financial inclusion is well maintained as first-class citizens but
> Mifos X isn't merely for microfinance and as we maintain a more modular
> platform, it will continue to be relevant for all those in the Fineract
> community looking to support various verticals for financial services.
>
> Ed
>
> On Sun, Aug 31, 2025 at 7:06 AM Paul <pc...@gmail.com> wrote:
>
>> Hi Felix,
>> My POV was directed for the whole community . . . and was not a
>> "counterpoint" to anyone.
>> Maybe an extension of thought and method? (why the broader Fineract
>> mission is good for unbanked and a "how to" example.).
>>
>>
>>
>> On Sun, Aug 31, 2025 at 7:41 AM Felix van Hove <fv...@gmx.de.invalid>
>> wrote:
>>
>>> Paul, I've thought a long time on your email. I think I find it
>>> difficult to answer, because - to a certain extend - I agree with you
>>> and James. It might be good to position Fineract broadly like this.
>>>
>>> But a discussion on a roadmap, with all parties involved, is of
>>> importance. To ask on the mailing list "Who is willing to fix a buggy
>>> feature X? Who will support it in the future?" and decommission it
>>> afterwards, because no one showed up, - appears too simple to me.
>>>
>>> Though I think I had more confidence in this happening, if I saw *any*
>>> new feature being proposed during my time, serving the same people that
>>> the Collection Sheet serves.
>>>
>>> Felix
>>>
>>>
>>>
>>> On 30/08/2025 22:49, Paul wrote:
>>> > Great feedback, Felix, Zayyad.
>>> > I would like to share a perspective from history and why the big
>>> picture
>>> > approach and broader mission is so important.
>>> >
>>> > There was a phrase in use when I was a baby. "Separate, but Equal."
>>> This
>>> > was a phrase used to justify keeping US school systems segregated by
>>> race.
>>> > "Separate but Equal" was a lie. The systems were never equal in
>>> funding,
>>> > quality or anything else.
>>> > Finance and access is the same.
>>> >
>>> > *What does that have to do with Fineract?*
>>> > Mission solely dedicated to the unbanked shrinks the number of people
>>> > willing to join the community.
>>> > Others may be sympathetic, but won't directly participate by
>>> contributing.
>>> > I happen to be interested in the unbanked and personally support that
>>> > mission.
>>> >
>>> > HOWEVER, my main interest in Fineract is to support a commercial
>>> enterprise
>>> > Open Source core banking system. A core banking system which integrates
>>> > embedds services anywhere, not just microfinance and money transfers.
>>> >
>>> > When the Fineract mission was focused on the unbanked, it was seriously
>>> > under participated IMO. With its slight change in mission which is
>>> still
>>> > inclusive of the underbanked, participation is increasing and releases
>>> are
>>> > beginning to become more regular. The unbanked needs to be supported
>>> with a
>>> > system capable of full integration with the rest of the world, not a
>>> > "separate but equal system".
>>> >
>>> > I hope my history story makes sense to at least support my POV.
>>> >
>>> > *How Fineract works for the unbanked with its current mission
>>> statement.  *
>>> >
>>> > MIFOS is solely dedicated to the underserved and has the frontend to
>>> do so.
>>> > In my opinion, if collection sheets are still used in working with
>>> > unbanked, then MIFOS community should adopt and help maintain them
>>> them as
>>> > a current, modern, secure function so that the MIFOS frontend works
>>> with
>>> > the Fineract backend. MIFOS like all other users of Fineract get the
>>> > benefit of a stronger core banking system and vendors with dedicated
>>> > purpose work together to maintain their particular core channels within
>>> > Fineract.
>>> >
>>> > The image is a generalization to convey the concept. (It may not be
>>> precise
>>> > as to the vendors and players, but should reinforce my POV.)
>>> > So, you, your interest or company wants a function or to maintain an
>>> > existing function? GREAT!
>>> > Step up! Own that solution within Fineract.
>>> > [image: image.png]
>>> > I speak solely for myself and my POV, but this approach will generate
>>> 10X
>>> > deeper support than segregated methods.
>>> >
>>> > In the past, there was insufficient community, but I think there was
>>> also
>>> > "community orchestration" missing to a degree.
>>> >
>>> > *Fail Model:*
>>> > Hoping it all gets done and telling the community at large "do
>>> something"
>>> > doesn't work.
>>> >
>>> > *Success Model: *
>>> > When a platform is useful across a whole ecosystem, then it can be
>>> jointly
>>> > maintained with nominal management. People are willing to help. Just
>>> like
>>> > musicians want to play music, they want to work together. In RARE
>>> > circumstances, the community is so SINGLE minded, the build, like JAZZ,
>>> > happens spontaneously and all agree, its really cool.
>>> >
>>> > *However, the bulk of the world needs direction and orchestration* to
>>> "make
>>> > music" of product delivery. Because of those many different interests,
>>> > Fineract needs process to harmonize and create results.
>>> >
>>> > In the end, there also must be a binary decision, "yes or no" to
>>> support an
>>> > issue. It comes down to; " will a community member support a function
>>> > important to their channel?"
>>> >
>>> > I'm not a developer, my useful skills are domain knowledge and
>>> > instructional orchestration (process). So I challenge all community
>>> members
>>> > to find a channel required to fulfill your channel or interest and
>>> > volunteer to "own it" through a date certain. Better yet, form a small
>>> team
>>> > of like minds and assure that function is ALWAYS current and secure.
>>> >
>>> > This is a challenge to all community members and targeting no one.
>>> >
>>> > Paul
>>> >
>>> > On Sat, Aug 30, 2025 at 3:34 PM James Dailey <jd...@apache.org>
>>> wrote:
>>> >
>>> >> Zayyad, Felix and others -
>>> >>
>>> >> Indeed.  And all of this was laid out for many years as the strategy.
>>> >>
>>> >> When I was on the Board of Mifos (the non profit, not the project) we
>>> >> decided to contribute the MifosX code to the ASF to ensure both:
>>> >>    a) a broader financial services industry approach, and
>>> >>    b) a more established foundational home for the core of the
>>> project.
>>> >>
>>> >> There were other reasons, but those remain primary in my mind.
>>> >>
>>> >> The concept was that Mifos org, which is a Vendor in the ASF parlance,
>>> >> would continue its Financial Inclusion mission and make sure that the
>>> >> releases that were coming out of the project would retain the flavor
>>> and
>>> >> intent of the original mifos project (which I started and designed in
>>> >> 2001).  The Mifos AdminUI /CommunityApp open source front end UI gave
>>> it a
>>> >> place in the ecosystem and a way to ensure that alignment.
>>> >>
>>> >> But, importantly, when we made this decision, we deliberately wanted
>>> to
>>> >> see more financial institutions and fintechs use the backend system,
>>> AND
>>> >> vitally, to help maintain it, with contributions upstream.  That is
>>> the
>>> >> sign of a healthy open source project.
>>> >>
>>> >> I don't speak for Mifos, but I believe that the vision at Mifos
>>> remains
>>> >> about financial inclusion.   But the vision of Fineract and the
>>> mission of
>>> >> the project is different.  We actually VOTED on this revised
>>> statement as a
>>> >> community, I think five years ago.
>>> >>
>>> >> So, if you are seeing a difference between Mifos and Fineract, then we
>>> >> have succeeded in an important goal.  The two are NOT THE SAME.  This
>>> is
>>> >> NOT mifos.
>>> >>
>>> >> This has been called out in many many Reports to the ASF Board.  See
>>> the
>>> >> wiki pages for those.  The key theme is Vendor neutrality - we do not
>>> >> privilege any vendor.
>>> >>
>>> >> And, I would argue that this is the right healthy direction for this
>>> >> Fineract project.  Microfinance serves an important role in the
>>> world, and
>>> >> continues despite many expecting it to decline or go away.  And, the
>>> >> mission of the original concept of Mifos was not about Microfinance
>>> but
>>> >> about financial inclusion.  However that happens.
>>> >>
>>> >> And, fundamentally, I personally remain focused on this - disrupting
>>> the
>>> >> financial industry with open source software that can functionally be
>>> the
>>> >> basis of any bank or fintech globally.  It should lower the cost of
>>> >> financial inclusion, it should do that via any kind of institution at
>>> a
>>> >> structural level.  Unbanked and underbanked are included.
>>> >>
>>> >> So, if you are not seeing what you want here, then you can invest more
>>> >> time and resources to make it more of what you want, but I would also
>>> >> suggest that the fundamental thing is to have healthy vendors that
>>> move the
>>> >> project forward with lots of "built on top of" functionality that
>>> serve a
>>> >> wide range of financial institutions.  If those components are also
>>> open
>>> >> source, that's ok, but it isn't essential.
>>> >>
>>> >> ** Deprecation process **
>>> >> Deprecating features is a natural part of any system evolution - we
>>> need
>>> >> to prune the tree from time to time.  If users are demanding those
>>> features
>>> >> then they should speak up - as this thread illustrated - and
>>> mechanisms
>>> >> need to be put in place to "protect them".  Here of course, I am
>>> referring
>>> >> to my last email and testing, to start with.
>>> >>
>>> >> And, we should do a better job of documenting the product roadmap, and
>>> >> having clear discussions about deprecating features - Before they are
>>> >> deprecated.
>>> >>
>>> >> That starts with you listing your required feature set and
>>> documenting it
>>> >> somewhere perhaps - or relying on the Vendor to speak up for your
>>> interests
>>> >> if you don't.
>>> >>
>>> >> And, this is a good discussion to have.  Perhaps a community call on
>>> this
>>> >> topic would be welcomed in coming weeks.
>>> >>
>>> >> James
>>> >>
>>> >>
>>> >>
>>> >> On Sat, Aug 30, 2025 at 3:13 AM Zayyad A. Said <
>>> >> zayyad@intrasofttechnologies.com> wrote:
>>> >>
>>> >>> Hello Fineract community,
>>> >>>
>>> >>> I concur with the observations below from Felix.
>>> >>>
>>> >>> We have been implementing Mifos since 2011 and we have seen the
>>> project
>>> >>> transitioning from Mifos 2.x to Mifos X and now with the new
>>> developments.
>>> >>> The current focus seems to have shifted from the original mission of
>>> >>> "Creating a world of 3 Billion Maries where each of the 2 billion
>>> poor and
>>> >>> unbanked has access to the financial resources needed to create a
>>> better
>>> >>> life for themselves and their family."
>>> >>>
>>> >>> Group lending has been used for decades and Mifos was originally
>>> intended
>>> >>> for and served the model, the new features being introduced are
>>> geared
>>> >>> towards commercial use. We saw this coming and we had to maintain
>>> old fork
>>> >>> for our MFI clients where we continue to improve based on their
>>> needs.
>>> >>>
>>> >>> While I have no problem with having new features, I seem not to
>>> >>> understand why we want to deprecate features of group lending that
>>> are
>>> >>> still being used even if with the minority of users globally yet
>>> they do no
>>> >>> harm to the security and overall performance of the project.
>>> >>>
>>> >>> I suggest the leadership to rethink the idea of deprecating
>>> collection
>>> >>> sheet functionality and other key features used in group lending.
>>> >>>
>>> >>> Best Regards,
>>> >>>
>>> >>> Zayyad A. Said
>>> >>> Intrasoft Technologies Limited
>>> >>>
>>> >>>   *********
>>> >>>
>>> >>> *Zayyad A. Said | Chief Executive Officer*
>>> >>>
>>> >>> Suite H32, Delamere Flats - Milimani Road, Nairobi.
>>> >>>
>>> >>> Cell No.: +254 716 615274 | Skype: *zsaid2011*
>>> >>>
>>> >>> Email: zayyad@intrasofttechnologies.com
>>> >>>
>>> >>> Schedule Meetings: https://calendly.com/zayyadsaid
>>> >>>
>>> >>> <https://calendly.com/zayyadsaid>
>>> >>>
>>> >>>
>>> >>> *" You can achieve what you want if you just help enough others get
>>> what
>>> >>> they want.."*
>>> >>>
>>> >>> *Note: Save the environment, please don't print this email unless its
>>> >>> really necessary for you to do that.*
>>> >>>
>>> >>> On Sat, Aug 30, 2025, 11:20 Felix van Hove <fv...@gmx.de.invalid>
>>> >>> wrote:
>>> >>>
>>> >>>> Let me chip in with a more personal perspective.
>>> >>>>
>>> >>>> I'm following the development of the Mifos web app and - to a
>>> certain
>>> >>>> extend - Fineract for a couple of months only. My impression from
>>> the
>>> >>>> influx of new features in Fineract is that these are predominantly
>>> not
>>> >>>> for microfinance or small entities, which Mifos once set out to
>>> support.
>>> >>>> In fact - Fineract only states that its efforts are also "including
>>> the
>>> >>>> unbanked and underbanked" - not its focus.
>>> >>>>
>>> >>>> I see the deprecation of the Collection Sheet feature in this
>>> context.
>>> >>>> How many new features have been added in the last 12 months that
>>> help
>>> >>>> the "unbanked"? How many have been added that help larger commercial
>>> >>>> entities and are of no benefit for "the unbanked"?
>>> >>>>
>>> >>>> My personal contributions depend solely on Mifos/Fineract supporting
>>> >>>> efforts for financial inclusion. I regard it as of utter importance
>>> >>>> respective features don't come as a plugin, but first class
>>> citizens.
>>> >>>>
>>> >>>> My knowledge regarding fintech is very limited. If I'm wrong or
>>> >>>> misrespresent things, please tell me.
>>> >>>>
>>> >>>> Felix
>>> >>>>
>>> >>>> On 29/08/2025 21:01, James Dailey wrote:
>>> >>>>> Devs -  Are we done with this?
>>> >>>>>
>>> >>>>> First, I am NOT ruling out this functionality as important but if
>>> no
>>> >>>> one is
>>> >>>>> actually demanding it, and no one is willing to maintain it, then
>>> it
>>> >>>>> doesn't get maintained.  That is the nature of open source
>>> projects.
>>> >>>>>
>>> >>>>> Second, it can happen that an outside vendor (e.g. Mifos) that
>>> relies
>>> >>>>> on this Collection Sheet functionality (or any functionality)
>>> >>>> inadvertently
>>> >>>>> drops that from their front end release.  Because the tests at
>>> >>>> Fineract do
>>> >>>>> not cover this fully, no one at the Vendor and no one at the
>>> Fineract
>>> >>>>> project will see the functionality fail in a build until a
>>> customer or
>>> >>>>> implementer notices. So, tests are vital for ensuring ongoing
>>> >>>>> maintainability.
>>> >>>>>
>>> >>>>> Third, if this type of thing isn't "naturally" part of the core -
>>> then
>>> >>>> it
>>> >>>>> will make a lot more sense to have it be an outside extension - in
>>> >>>> which
>>> >>>>> case the tests have to be written for the API calls, and a Vendor
>>> >>>> should
>>> >>>>> try to get such tests contributed.   I could be wrong about this, I
>>> >>>> would
>>> >>>>> be happy to debate where this belongs.
>>> >>>>>
>>> >>>>> @Edward Cable <ed...@mifos.org> - what is the assessment and
>>> status
>>> >>>> of
>>> >>>>> needed functionality and where do you think it should live?
>>> >>>>>
>>> >>>>> Thanks,
>>> >>>>> James
>>> >>>>>
>>> >>>>> On Mon, Aug 18, 2025 at 8:21 AM James Dailey <jd...@apache.org>
>>> >>>> wrote:
>>> >>>>>
>>> >>>>>> Sifiso -
>>> >>>>>>
>>> >>>>>> Do we have a way to reach the tens of thousands of institutions
>>> you
>>> >>>>>> believe are there?
>>> >>>>>>
>>> >>>>>> My belief (just anecdotal information) is that it’s more like a
>>> few
>>> >>>>>> hundred institutions and that out of that number, less than 10%
>>> are
>>> >>>> on a
>>> >>>>>> recent release.  And, of that smaller number, few, if any, are
>>> still
>>> >>>> using
>>> >>>>>> Collection Sheets.  I could be wrong - I’d like the data!  I’d say
>>> >>>> that
>>> >>>>>> vendors can show up with their data here to help make this vendor
>>> >>>> neutral
>>> >>>>>> space more informed.
>>> >>>>>>
>>> >>>>>> If group lending and group-member lending and collection sheets
>>> are
>>> >>>> still
>>> >>>>>> needed, then perhaps those project volunteers can contribute the
>>> >>>> needed
>>> >>>>>> updates to keep the functionality useful.
>>> >>>>>>
>>> >>>>>> Thanks
>>> >>>>>> Thanks
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> On Mon, Aug 18, 2025 at 6:48 AM Sifiso Mtetwa <
>>> >>>> sifiso@skyburgsystems.org>
>>> >>>>>> wrote:
>>> >>>>>>
>>> >>>>>>> Hi guys,
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> This is an interesting topic. I have been wondering why, in
>>> general,
>>> >>>> we
>>> >>>>>>> seem to be deprecating more and more system functions. The
>>> individual
>>> >>>>>>> collection sheet has served us well over the years and still
>>> does, If
>>> >>>>>>> anything we could improve on its functionality by maybe adding a
>>> bulk
>>> >>>>>>> collection sheet template with little detail compared to a full
>>> loan
>>> >>>>>>> repayment bulk import template.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> Fineract is used by tens of thousands of organisations
>>> throughout the
>>> >>>>>>> world and most of them are not on this listing and may not have a
>>> >>>> voice to
>>> >>>>>>> air their concerns. Maybe we can find a way of exposing this
>>> thread
>>> >>>> to
>>> >>>>>>> include more voters.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> Regards,
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> *From:* James Dailey [mailto:jdailey@apache.org]
>>> >>>>>>> *Sent:* Saturday, 16 August 2025 02:53
>>> >>>>>>> *To:* dev@fineract.apache.org
>>> >>>>>>> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions
>>> and
>>> >>>>>>> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> Ed
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> I agree this needs to be discussed but it is important to
>>> acknowledge
>>> >>>>>>> that THIS Fineract listserv is the only official (and required)
>>> >>>> discussion
>>> >>>>>>> space.  It is not the intention to bury anything in “an email
>>> >>>> thread”.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> When I started up Mifos we spent a lot of time looking at
>>> Collection
>>> >>>>>>> Sheets and designing process flows around them. I fully know that
>>> >>>> this
>>> >>>>>>> design direction was important back then in 2002-2006.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> However if there is no one here asking for them to be retained
>>> >>>> besides
>>> >>>>>>> you, then that is a sign that they have perhaps reached an end of
>>> >>>> their
>>> >>>>>>> utility.  Or, that the users are not actually here, which is a
>>> >>>> different
>>> >>>>>>> problem.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> So,… If there is a group using collection sheets in production
>>> AND
>>> >>>> they
>>> >>>>>>> are not on some permanent forked (old) version, then now is the
>>> time
>>> >>>> to
>>> >>>>>>> speak up.  Here.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> Generally, I’m fairly certain we can refactor this with an eye
>>> toward
>>> >>>>>>> extracting it from the core.  Repeating the logic in two places
>>> >>>> makes no
>>> >>>>>>> sense either.  Collection sheets are kind of assembled from
>>> >>>>>>> constituent loans and savings.  The balances and due payments
>>> should
>>> >>>> be
>>> >>>>>>> calculated in the underlying components.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> As the system gets restructured we need to decide to keep this at
>>> >>>> all, to
>>> >>>>>>> keep it in a new place, or as some external concept/plug in. Why
>>> >>>> wouldn’t
>>> >>>>>>> we want a separate component?
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> Cheers
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> On Thu, Aug 14, 2025 at 4:16 PM Ed Cable <ed...@mifos.org>
>>> wrote:
>>> >>>>>>>
>>> >>>>>>> I'll leave the other thread for discussion of the API versioning
>>> and
>>> >>>>>>> refactoring related to Collection Sheet API but wanted to create
>>> a
>>> >>>> separate
>>> >>>>>>> thread regarding the deprecation of the Collection Sheet.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> In general, for this and removal of any functionality, it's
>>> something
>>> >>>>>>> that needs to be discussed openly with the community and with a
>>> vote
>>> >>>> and
>>> >>>>>>> not a decision buried in a mailing list thread.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> For the collection sheet specifically, more thought has to be
>>> given
>>> >>>> to
>>> >>>>>>> its deprecation as the centrality and highly coupled nature of
>>> the
>>> >>>>>>> collection sheet is being understated as it isn't merely a report
>>> >>>> that's to
>>> >>>>>>> be printed or a form filled out via a mobile application. It's a
>>> >>>>>>> significant portion of the user interface and highly coupled to
>>> many
>>> >>>> of the
>>> >>>>>>> microfinance features around groups/centers/meeting scheduling,
>>> staff
>>> >>>>>>> assignment, etc.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> I do agree that from a UI perspective, the collection sheet and
>>> other
>>> >>>>>>> microfinance-centric functionalities and flows should be viewable
>>> >>>> based on
>>> >>>>>>> a configurable setting. As it doesn't lend itself to the optimal
>>> user
>>> >>>>>>> experience for a large portion of current Fineract user base.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> I also am supportive of a strategy of slimming Fineract to its
>>> core
>>> >>>>>>> services and functionality above core Fineract services and APIs
>>> can
>>> >>>> be
>>> >>>>>>> extracted out.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> So I do think we should give thoughtful consideration to what
>>> >>>> abstracting
>>> >>>>>>> out the collection sheet and corresponding microfinance
>>> >>>> functionality would
>>> >>>>>>> look like and what that effort would entail to abstract it out
>>> >>>> without
>>> >>>>>>> adversely impacting the original user base of the software but
>>> it's
>>> >>>> not as
>>> >>>>>>> simple as deprecating these API.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> I welcome others' thoughts and inputs as I know even with
>>> >>>> microfinance
>>> >>>>>>> itself, the methodology has evolved and group lending and the
>>> >>>> concept of a
>>> >>>>>>> collection sheet isn't as central as it once was
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> Thanks,
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> Ed
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> On Wed, Aug 13, 2025 at 6:36 PM Kapil Panchal <
>>> >>>>>>> kapil.panchal.development@gmail.com> wrote:
>>> >>>>>>>
>>> >>>>>>> Hi James,
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> I can proceed by marking this feature as @Deprecated and/or
>>> >>>> performing a
>>> >>>>>>> safe refactor to remove the API.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> For API versioning I agree to Aleksandar Vidakovic's proposal on
>>> >>>> adapting
>>> >>>>>>> to the SpringBoot v7/Spring Framework v4. If I may, Aleksandar
>>> >>>>>>> Vidakovic takes the lead on this project and I can help to
>>> support
>>> >>>> the
>>> >>>>>>> conversion?
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> Thanks,
>>> >>>>>>>
>>> >>>>>>> Kapil
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> On Thu, Aug 14, 2025 at 5:07 AM James Dailey <jdailey@apache.org
>>> >
>>> >>>> wrote:
>>> >>>>>>>
>>> >>>>>>> Hi Kapil
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> I might suggest looking at this as an opportunity to remove the
>>> >>>>>>> collection sheet entirely from the Fineract namespace.  It’s a
>>> legacy
>>> >>>>>>> concept I and others designed a long time ago, originally in 2002
>>> >>>> based on
>>> >>>>>>> collection sheets we gathered from a dozen countries. It is
>>> strongly
>>> >>>> tied
>>> >>>>>>> to concepts in microfinance field operations, and especially when
>>> >>>> there was
>>> >>>>>>> no data connectivity.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> It belongs perhaps as a sort of external microservice - data
>>> loading
>>> >>>> via
>>> >>>>>>> a bulk import could still be enabled.
>>> >>>>>>>
>>> >>>>>>> The API versioning is a good idea but needs to be more holistic
>>> >>>> across
>>> >>>>>>> the platform I think.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <ad...@gmail.com>
>>> >>>> wrote:
>>> >>>>>>>
>>> >>>>>>> Hi Kapil,
>>> >>>>>>>
>>> >>>>>>> Thank you for raising the concerns below. I’ll need some
>>> additional
>>> >>>>>>> details to fully understand your points:
>>> >>>>>>>
>>> >>>>>>> 1.      *Collection Sheet API* – You mentioned it appears
>>> >>>> non-functional
>>> >>>>>>> and contains several logical errors.
>>> >>>>>>>
>>> >>>>>>> o    If it’s indeed not working, that’s a separate, high-priority
>>> >>>>>>> discussion.
>>> >>>>>>>
>>> >>>>>>> o    Could you clarify which logical errors you were referring
>>> to,
>>> >>>> and
>>> >>>>>>> what specifically makes you think it’s non-functional?
>>> >>>>>>>
>>> >>>>>>> 2.      *Service annotations* – You noted that service methods
>>> are
>>> >>>> not
>>> >>>>>>> annotated with @Service and that beans are defined manually.
>>> >>>>>>>
>>> >>>>>>> o    Are you referring to the
>>> >>>>>>> CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being
>>> >>>> defined
>>> >>>>>>> via configuration?
>>> >>>>>>>
>>> >>>>>>> 3.      *Repository wrappers annotated with **@Service* – You
>>> >>>> mentioned
>>> >>>>>>> that this mandates full unit test coverage but that they should
>>> >>>> ideally be
>>> >>>>>>> annotated with @Component.
>>> >>>>>>>
>>> >>>>>>> o    Could you point out the exact classes you had in mind?
>>> >>>>>>>
>>> >>>>>>> As for the other points, I agree we can refactor and remove
>>> redundant
>>> >>>>>>> logic—please feel free to suggest specific improvements or start
>>> >>>> work on
>>> >>>>>>> them immediately!
>>> >>>>>>>
>>> >>>>>>> However, be careful by moving anything into the fineract-core…
>>> We are
>>> >>>>>>> aiming to keep it as small as possible as everything is built on
>>> top
>>> >>>> of
>>> >>>>>>> this module! If collection sheet are used for loans and savings
>>> - for
>>> >>>>>>> example - than the recommended move is NOT to move this logic
>>> into
>>> >>>> core!
>>> >>>>>>>
>>> >>>>>>> Either:
>>> >>>>>>>
>>> >>>>>>> - we split the logic into fineract-loan and fineract-savings
>>> >>>>>>>
>>> >>>>>>> - Move the logic into a new module
>>> >>>>>>>
>>> >>>>>>> - Leave it in fineract-provider for now
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> Shall you have any questions, please let us know!
>>> >>>>>>>
>>> >>>>>>> Regards,
>>> >>>>>>> Adam
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> On 2025. Aug 11., at 12:09, Kapil Panchal <
>>> >>>>>>> kapil.panchal.development@gmail.com> wrote:
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> Hi Adam,
>>> >>>>>>>
>>> >>>>>>> I’m currently working on *FINERACT-2290* and have a few questions
>>> >>>> before
>>> >>>>>>> I submit a pull request.
>>> >>>>>>>
>>> >>>>>>> The *Collection Sheet API* in its current state appears
>>> >>>> non-functional
>>> >>>>>>> and contains several logical errors. It seems there was an
>>> earlier
>>> >>>> attempt
>>> >>>>>>> to convert from a JSON string request parameter to a class-based
>>> >>>> request
>>> >>>>>>> object, but:
>>> >>>>>>>
>>> >>>>>>> Certain fields are missing.
>>> >>>>>>>
>>> >>>>>>> The serializer is not correctly populating the objects, which
>>> causes
>>> >>>> the
>>> >>>>>>> conditional checks to be bypassed and results in incorrect
>>> (false)
>>> >>>>>>> responses.
>>> >>>>>>>
>>> >>>>>>> This change set is *high risk* because it touches most of the
>>> loan
>>> >>>> and
>>> >>>>>>> savings product logic. I’ve had to refactor almost all major
>>> methods.
>>> >>>>>>> Extensive integration and end-to-end testing will be required to
>>> >>>> ensure
>>> >>>>>>> there are no regressions, especially in edge cases. At present,
>>> >>>> there are
>>> >>>>>>> no unit or integration tests for this functionality, and test
>>> >>>> creation is
>>> >>>>>>> outside the current ticket scope. I’ve been iterating on this
>>> for a
>>> >>>> while,
>>> >>>>>>> and only today have I reached a stable state after several
>>> >>>> experimental and
>>> >>>>>>> build-breaking attempts.
>>> >>>>>>>
>>> >>>>>>> *Key Observations:*
>>> >>>>>>>
>>> >>>>>>> Service methods are not annotated with @Service; instead, beans
>>> are
>>> >>>>>>> defined manually.
>>> >>>>>>>
>>> >>>>>>> Repository wrappers are annotated with @Service. This mandates
>>> full
>>> >>>> unit
>>> >>>>>>> test coverage for these methods, but they should ideally be
>>> >>>> annotated with
>>> >>>>>>> @Component.
>>> >>>>>>>
>>> >>>>>>> I agree with prior discussions on separating bean validation —
>>> >>>> having a
>>> >>>>>>> dedicated @Component validation class allows the request object
>>> to
>>> >>>> handle
>>> >>>>>>> checks independent of database queries.
>>> >>>>>>>
>>> >>>>>>> Validation components can also perform database-related
>>> validations;
>>> >>>>>>> these can be injected into service classes for cleaner
>>> architecture.
>>> >>>>>>>
>>> >>>>>>> Such validation components should be placed in *Fineract-Core* so
>>> >>>> they
>>> >>>>>>> are reusable across modules, reducing future refactoring needs.
>>> >>>>>>>
>>> >>>>>>> The current design of having commands in *Fineract-Core* and
>>> >>>>>>> handlers/services/repositories in respective modules is good — it
>>> >>>> cleanly
>>> >>>>>>> decouples command definition from execution.
>>> >>>>>>>
>>> >>>>>>> There is extensive use of this. in singleton contexts (API,
>>> Service,
>>> >>>>>>> Repository). While not harmful, it’s unnecessary boilerplate.
>>> >>>>>>>
>>> >>>>>>> Multiple redundant intermediate DTOs exist where the request DTO
>>> >>>> itself
>>> >>>>>>> could be reused for data transfer.
>>> >>>>>>>
>>> >>>>>>> I found redundant logic — e.g., a for loop with a break statement
>>> >>>> that
>>> >>>>>>> effectively executes only once; this can be simplified.
>>> >>>>>>>
>>> >>>>>>> Some JDBC template queries use reserved SQL keywords, causing
>>> >>>> exceptions.
>>> >>>>>>> Refactoring these queries resolves the issue and returns proper
>>> >>>> response
>>> >>>>>>> objects.
>>> >>>>>>>
>>> >>>>>>> *Suggestions:*
>>> >>>>>>>
>>> >>>>>>> *Where appropriate, large tickets should be broken into subtasks
>>> to
>>> >>>>>>> manage complexity and reviewability.*
>>> >>>>>>>
>>> >>>>>>> It may help to have a dedicated *developer-only Slack channel
>>> *for
>>> >>>>>>> technical discussions. This could complement other community
>>> spaces
>>> >>>> if
>>> >>>>>>> there’s a need to keep certain conversations more focused.
>>> >>>>>>>
>>> >>>>>>> What are your thoughts on the above?
>>> >>>>>>>
>>> >>>>>>> Thanks,
>>> >>>>>>> Kapil
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> --
>>> >>>>>>>
>>> >>>>>>> *Ed Cable*
>>> >>>>>>>
>>> >>>>>>> President/CEO, Mifos Initiative
>>> >>>>>>>
>>> >>>>>>> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> *Collectively Creating a World of 3 Billion Maries | *
>>> >>>> http://mifos.org
>>> >>>>>>> <http://facebook.com/mifos>  <http://www.twitter.com/mifos>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>
>>> >>>>
>>> >>>>
>>> >
>>>
>>>
>>
>> --
>> --
>> Paul
>>
>
>
> --
> *Ed Cable*
> President/CEO, Mifos Initiative
> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>
> *Collectively Creating a World of 3 Billion Maries | *http://mifos.org
> <http://facebook.com/mifos>  <http://www.twitter.com/mifos>
>
>

Re: Mission of Fineract discussion (was Re: Collection Sheet Deprecation ...)

[Ed Cable <ed...@mifos.org>]

Thank you everyone for this valuable discussion and the inputs thus far.

To assuage the concerns of Zayyad, Sifiso, Felix and others, Mifos,
regardless of what decision the Fineract community makes in regards to
deprecation of collection sheet functionality, Mifos will not be removing
this functionality nor diminishing its focus on microfinance and financial
inclusion within the Mifos X distribution nor within the Mifos X community.
This is at both the back-end as well as front-end UI level, and our goal is
to make these features more configurable via the UI so they can be
supported from the back-end but not cluttering the user experience for
those who don't need it. I'll make sure those who are working on
maintaining these key pieces of functionality are sharing regular updates
via community channels.

I agree with Paul's articulation of the broader umbrella and bigger tent
that Fineract helps to provide as well as the need for various
individuals in the community representing certain verticals of financial
services using the software to be more participatory in the upstream
communities (ideally this would be in the form of code contribution, but at
minimum it could be staying on the latest versions of the codebase, and
providing active and engaged feedback in terms of new requirements, bug
reports, usability feedback, etc.) I know that over the years Mifos has
struggled to get proper user feedback for intern projects for
requirements/features that do relate to microfinance.

*So I think one key point is if the community wants its voice heard and the
respective project to meet their ongoing requirements, they need to be
engaged, active, and participatory* - I recognize that the mailing list
might not be that best place for many of the users so that's why Mifos has
an active Slack channel, has experimented with other communication
tools/formats, and hosts regular community calls, and engages one on one
with vendors representing these verticals.

Concerning the focus on new features and functionality for microfinance and
financial inclusion, part of the reason why there has been so little new
development here has been the silver lining of this feature set being
mature and near fully-functional. As history recounts, JLG lending was the
original use case for Mifos, expanding to a broader feature set beyond
lending for cooperatives and SACCOs and more of the organic contribution
that has occurred with more advanced lending and formal financial services.
That being said, Mifos is working with members of the ecosystem and
community to enhance teller operations, better support savings-led
approaches like the VSLA methodology, and better Shariah-compliant products
for microfinance to name a few.

I don't think we should view the sponsored contributions of more advanced
lending features that Felix has highlighted as dominating the roadmap nor
diminishing the platform's relevance and impact for microfinance and
financial inclusion. Mifos has been shepherding this commercial project
yielding these contributions over the past several years and we've been
very deliberate about making the functionality as generic as possible and
much of the functionality does add benefit to all users of the software.
However these enhancements and how they can be configured could be better
documented which is underway. Beyond the upstream code and the strong
reference this customer provides, the Pepper Soup project has had many
other benefits for the sustainability of the organizations stewarding the
community to ensure that the focus can remain on the mission, timely
releases are made, etc.

A main reason why these contributions seem to overshadow others goes back
to the point Paul raises is that others aren't contributing and the
community needs to step up and contribute and participate upstream. This
customer made the conscious decision and commitment to contribute all its
enhancements upstream. If others in the community were willing to do the
same with the features they've been building in their downstream solutions,
we'd see a more balanced flow of contribution upstream. There are no
barriers for these contributions coming in as long as they follow the
community process. Mifos recognizes this requires a bit of handholding and
we continue to try to enable more individuals and vendors to contribute
upstream.

Felix does raise a good point around the process and visibility into a
roadmap and that is something James has been more proactive about with
Fineract and Mifos will be doing as well in regards to Mifos X to provide
the forward-looking view of the software, gather inputs on where others
would like to contribute, highlight gaps that need closing, and signal to
the community where help is needed in contributing to close these gaps.

A last point I'll make is recognizing that if we are successful in both
maintaining the existing feature set and merging in more upstream
contributions to support the wide number of verticals that Mifos and
Fineract attracts, the user experience of the software does get complicated
and overwhelming and the software can be hard to maintain. Mifos is
committed to ongoing modularity at both the front and back-end and is
advancing efforts led by Aleks Vidakovic to achieve the same.

So rest assured that Mifos will ensure the feature set for microfinance and
financial inclusion is well maintained as first-class citizens but Mifos X
isn't merely for microfinance and as we maintain a more modular platform,
it will continue to be relevant for all those in the Fineract community
looking to support various verticals for financial services.

Ed

On Sun, Aug 31, 2025 at 7:06 AM Paul <pc...@gmail.com> wrote:

> Hi Felix,
> My POV was directed for the whole community . . . and was not a
> "counterpoint" to anyone.
> Maybe an extension of thought and method? (why the broader Fineract
> mission is good for unbanked and a "how to" example.).
>
>
>
> On Sun, Aug 31, 2025 at 7:41 AM Felix van Hove <fv...@gmx.de.invalid>
> wrote:
>
>> Paul, I've thought a long time on your email. I think I find it
>> difficult to answer, because - to a certain extend - I agree with you
>> and James. It might be good to position Fineract broadly like this.
>>
>> But a discussion on a roadmap, with all parties involved, is of
>> importance. To ask on the mailing list "Who is willing to fix a buggy
>> feature X? Who will support it in the future?" and decommission it
>> afterwards, because no one showed up, - appears too simple to me.
>>
>> Though I think I had more confidence in this happening, if I saw *any*
>> new feature being proposed during my time, serving the same people that
>> the Collection Sheet serves.
>>
>> Felix
>>
>>
>>
>> On 30/08/2025 22:49, Paul wrote:
>> > Great feedback, Felix, Zayyad.
>> > I would like to share a perspective from history and why the big picture
>> > approach and broader mission is so important.
>> >
>> > There was a phrase in use when I was a baby. "Separate, but Equal." This
>> > was a phrase used to justify keeping US school systems segregated by
>> race.
>> > "Separate but Equal" was a lie. The systems were never equal in funding,
>> > quality or anything else.
>> > Finance and access is the same.
>> >
>> > *What does that have to do with Fineract?*
>> > Mission solely dedicated to the unbanked shrinks the number of people
>> > willing to join the community.
>> > Others may be sympathetic, but won't directly participate by
>> contributing.
>> > I happen to be interested in the unbanked and personally support that
>> > mission.
>> >
>> > HOWEVER, my main interest in Fineract is to support a commercial
>> enterprise
>> > Open Source core banking system. A core banking system which integrates
>> > embedds services anywhere, not just microfinance and money transfers.
>> >
>> > When the Fineract mission was focused on the unbanked, it was seriously
>> > under participated IMO. With its slight change in mission which is still
>> > inclusive of the underbanked, participation is increasing and releases
>> are
>> > beginning to become more regular. The unbanked needs to be supported
>> with a
>> > system capable of full integration with the rest of the world, not a
>> > "separate but equal system".
>> >
>> > I hope my history story makes sense to at least support my POV.
>> >
>> > *How Fineract works for the unbanked with its current mission
>> statement.  *
>> >
>> > MIFOS is solely dedicated to the underserved and has the frontend to do
>> so.
>> > In my opinion, if collection sheets are still used in working with
>> > unbanked, then MIFOS community should adopt and help maintain them them
>> as
>> > a current, modern, secure function so that the MIFOS frontend works with
>> > the Fineract backend. MIFOS like all other users of Fineract get the
>> > benefit of a stronger core banking system and vendors with dedicated
>> > purpose work together to maintain their particular core channels within
>> > Fineract.
>> >
>> > The image is a generalization to convey the concept. (It may not be
>> precise
>> > as to the vendors and players, but should reinforce my POV.)
>> > So, you, your interest or company wants a function or to maintain an
>> > existing function? GREAT!
>> > Step up! Own that solution within Fineract.
>> > [image: image.png]
>> > I speak solely for myself and my POV, but this approach will generate
>> 10X
>> > deeper support than segregated methods.
>> >
>> > In the past, there was insufficient community, but I think there was
>> also
>> > "community orchestration" missing to a degree.
>> >
>> > *Fail Model:*
>> > Hoping it all gets done and telling the community at large "do
>> something"
>> > doesn't work.
>> >
>> > *Success Model: *
>> > When a platform is useful across a whole ecosystem, then it can be
>> jointly
>> > maintained with nominal management. People are willing to help. Just
>> like
>> > musicians want to play music, they want to work together. In RARE
>> > circumstances, the community is so SINGLE minded, the build, like JAZZ,
>> > happens spontaneously and all agree, its really cool.
>> >
>> > *However, the bulk of the world needs direction and orchestration* to
>> "make
>> > music" of product delivery. Because of those many different interests,
>> > Fineract needs process to harmonize and create results.
>> >
>> > In the end, there also must be a binary decision, "yes or no" to
>> support an
>> > issue. It comes down to; " will a community member support a function
>> > important to their channel?"
>> >
>> > I'm not a developer, my useful skills are domain knowledge and
>> > instructional orchestration (process). So I challenge all community
>> members
>> > to find a channel required to fulfill your channel or interest and
>> > volunteer to "own it" through a date certain. Better yet, form a small
>> team
>> > of like minds and assure that function is ALWAYS current and secure.
>> >
>> > This is a challenge to all community members and targeting no one.
>> >
>> > Paul
>> >
>> > On Sat, Aug 30, 2025 at 3:34 PM James Dailey <jd...@apache.org>
>> wrote:
>> >
>> >> Zayyad, Felix and others -
>> >>
>> >> Indeed.  And all of this was laid out for many years as the strategy.
>> >>
>> >> When I was on the Board of Mifos (the non profit, not the project) we
>> >> decided to contribute the MifosX code to the ASF to ensure both:
>> >>    a) a broader financial services industry approach, and
>> >>    b) a more established foundational home for the core of the project.
>> >>
>> >> There were other reasons, but those remain primary in my mind.
>> >>
>> >> The concept was that Mifos org, which is a Vendor in the ASF parlance,
>> >> would continue its Financial Inclusion mission and make sure that the
>> >> releases that were coming out of the project would retain the flavor
>> and
>> >> intent of the original mifos project (which I started and designed in
>> >> 2001).  The Mifos AdminUI /CommunityApp open source front end UI gave
>> it a
>> >> place in the ecosystem and a way to ensure that alignment.
>> >>
>> >> But, importantly, when we made this decision, we deliberately wanted to
>> >> see more financial institutions and fintechs use the backend system,
>> AND
>> >> vitally, to help maintain it, with contributions upstream.  That is the
>> >> sign of a healthy open source project.
>> >>
>> >> I don't speak for Mifos, but I believe that the vision at Mifos remains
>> >> about financial inclusion.   But the vision of Fineract and the
>> mission of
>> >> the project is different.  We actually VOTED on this revised statement
>> as a
>> >> community, I think five years ago.
>> >>
>> >> So, if you are seeing a difference between Mifos and Fineract, then we
>> >> have succeeded in an important goal.  The two are NOT THE SAME.  This
>> is
>> >> NOT mifos.
>> >>
>> >> This has been called out in many many Reports to the ASF Board.  See
>> the
>> >> wiki pages for those.  The key theme is Vendor neutrality - we do not
>> >> privilege any vendor.
>> >>
>> >> And, I would argue that this is the right healthy direction for this
>> >> Fineract project.  Microfinance serves an important role in the world,
>> and
>> >> continues despite many expecting it to decline or go away.  And, the
>> >> mission of the original concept of Mifos was not about Microfinance but
>> >> about financial inclusion.  However that happens.
>> >>
>> >> And, fundamentally, I personally remain focused on this - disrupting
>> the
>> >> financial industry with open source software that can functionally be
>> the
>> >> basis of any bank or fintech globally.  It should lower the cost of
>> >> financial inclusion, it should do that via any kind of institution at a
>> >> structural level.  Unbanked and underbanked are included.
>> >>
>> >> So, if you are not seeing what you want here, then you can invest more
>> >> time and resources to make it more of what you want, but I would also
>> >> suggest that the fundamental thing is to have healthy vendors that
>> move the
>> >> project forward with lots of "built on top of" functionality that
>> serve a
>> >> wide range of financial institutions.  If those components are also
>> open
>> >> source, that's ok, but it isn't essential.
>> >>
>> >> ** Deprecation process **
>> >> Deprecating features is a natural part of any system evolution - we
>> need
>> >> to prune the tree from time to time.  If users are demanding those
>> features
>> >> then they should speak up - as this thread illustrated - and mechanisms
>> >> need to be put in place to "protect them".  Here of course, I am
>> referring
>> >> to my last email and testing, to start with.
>> >>
>> >> And, we should do a better job of documenting the product roadmap, and
>> >> having clear discussions about deprecating features - Before they are
>> >> deprecated.
>> >>
>> >> That starts with you listing your required feature set and documenting
>> it
>> >> somewhere perhaps - or relying on the Vendor to speak up for your
>> interests
>> >> if you don't.
>> >>
>> >> And, this is a good discussion to have.  Perhaps a community call on
>> this
>> >> topic would be welcomed in coming weeks.
>> >>
>> >> James
>> >>
>> >>
>> >>
>> >> On Sat, Aug 30, 2025 at 3:13 AM Zayyad A. Said <
>> >> zayyad@intrasofttechnologies.com> wrote:
>> >>
>> >>> Hello Fineract community,
>> >>>
>> >>> I concur with the observations below from Felix.
>> >>>
>> >>> We have been implementing Mifos since 2011 and we have seen the
>> project
>> >>> transitioning from Mifos 2.x to Mifos X and now with the new
>> developments.
>> >>> The current focus seems to have shifted from the original mission of
>> >>> "Creating a world of 3 Billion Maries where each of the 2 billion
>> poor and
>> >>> unbanked has access to the financial resources needed to create a
>> better
>> >>> life for themselves and their family."
>> >>>
>> >>> Group lending has been used for decades and Mifos was originally
>> intended
>> >>> for and served the model, the new features being introduced are geared
>> >>> towards commercial use. We saw this coming and we had to maintain old
>> fork
>> >>> for our MFI clients where we continue to improve based on their needs.
>> >>>
>> >>> While I have no problem with having new features, I seem not to
>> >>> understand why we want to deprecate features of group lending that are
>> >>> still being used even if with the minority of users globally yet they
>> do no
>> >>> harm to the security and overall performance of the project.
>> >>>
>> >>> I suggest the leadership to rethink the idea of deprecating collection
>> >>> sheet functionality and other key features used in group lending.
>> >>>
>> >>> Best Regards,
>> >>>
>> >>> Zayyad A. Said
>> >>> Intrasoft Technologies Limited
>> >>>
>> >>>   *********
>> >>>
>> >>> *Zayyad A. Said | Chief Executive Officer*
>> >>>
>> >>> Suite H32, Delamere Flats - Milimani Road, Nairobi.
>> >>>
>> >>> Cell No.: +254 716 615274 | Skype: *zsaid2011*
>> >>>
>> >>> Email: zayyad@intrasofttechnologies.com
>> >>>
>> >>> Schedule Meetings: https://calendly.com/zayyadsaid
>> >>>
>> >>> <https://calendly.com/zayyadsaid>
>> >>>
>> >>>
>> >>> *" You can achieve what you want if you just help enough others get
>> what
>> >>> they want.."*
>> >>>
>> >>> *Note: Save the environment, please don't print this email unless its
>> >>> really necessary for you to do that.*
>> >>>
>> >>> On Sat, Aug 30, 2025, 11:20 Felix van Hove <fv...@gmx.de.invalid>
>> >>> wrote:
>> >>>
>> >>>> Let me chip in with a more personal perspective.
>> >>>>
>> >>>> I'm following the development of the Mifos web app and - to a certain
>> >>>> extend - Fineract for a couple of months only. My impression from the
>> >>>> influx of new features in Fineract is that these are predominantly
>> not
>> >>>> for microfinance or small entities, which Mifos once set out to
>> support.
>> >>>> In fact - Fineract only states that its efforts are also "including
>> the
>> >>>> unbanked and underbanked" - not its focus.
>> >>>>
>> >>>> I see the deprecation of the Collection Sheet feature in this
>> context.
>> >>>> How many new features have been added in the last 12 months that help
>> >>>> the "unbanked"? How many have been added that help larger commercial
>> >>>> entities and are of no benefit for "the unbanked"?
>> >>>>
>> >>>> My personal contributions depend solely on Mifos/Fineract supporting
>> >>>> efforts for financial inclusion. I regard it as of utter importance
>> >>>> respective features don't come as a plugin, but first class citizens.
>> >>>>
>> >>>> My knowledge regarding fintech is very limited. If I'm wrong or
>> >>>> misrespresent things, please tell me.
>> >>>>
>> >>>> Felix
>> >>>>
>> >>>> On 29/08/2025 21:01, James Dailey wrote:
>> >>>>> Devs -  Are we done with this?
>> >>>>>
>> >>>>> First, I am NOT ruling out this functionality as important but if no
>> >>>> one is
>> >>>>> actually demanding it, and no one is willing to maintain it, then it
>> >>>>> doesn't get maintained.  That is the nature of open source projects.
>> >>>>>
>> >>>>> Second, it can happen that an outside vendor (e.g. Mifos) that
>> relies
>> >>>>> on this Collection Sheet functionality (or any functionality)
>> >>>> inadvertently
>> >>>>> drops that from their front end release.  Because the tests at
>> >>>> Fineract do
>> >>>>> not cover this fully, no one at the Vendor and no one at the
>> Fineract
>> >>>>> project will see the functionality fail in a build until a customer
>> or
>> >>>>> implementer notices. So, tests are vital for ensuring ongoing
>> >>>>> maintainability.
>> >>>>>
>> >>>>> Third, if this type of thing isn't "naturally" part of the core -
>> then
>> >>>> it
>> >>>>> will make a lot more sense to have it be an outside extension - in
>> >>>> which
>> >>>>> case the tests have to be written for the API calls, and a Vendor
>> >>>> should
>> >>>>> try to get such tests contributed.   I could be wrong about this, I
>> >>>> would
>> >>>>> be happy to debate where this belongs.
>> >>>>>
>> >>>>> @Edward Cable <ed...@mifos.org> - what is the assessment and
>> status
>> >>>> of
>> >>>>> needed functionality and where do you think it should live?
>> >>>>>
>> >>>>> Thanks,
>> >>>>> James
>> >>>>>
>> >>>>> On Mon, Aug 18, 2025 at 8:21 AM James Dailey <jd...@apache.org>
>> >>>> wrote:
>> >>>>>
>> >>>>>> Sifiso -
>> >>>>>>
>> >>>>>> Do we have a way to reach the tens of thousands of institutions you
>> >>>>>> believe are there?
>> >>>>>>
>> >>>>>> My belief (just anecdotal information) is that it’s more like a few
>> >>>>>> hundred institutions and that out of that number, less than 10% are
>> >>>> on a
>> >>>>>> recent release.  And, of that smaller number, few, if any, are
>> still
>> >>>> using
>> >>>>>> Collection Sheets.  I could be wrong - I’d like the data!  I’d say
>> >>>> that
>> >>>>>> vendors can show up with their data here to help make this vendor
>> >>>> neutral
>> >>>>>> space more informed.
>> >>>>>>
>> >>>>>> If group lending and group-member lending and collection sheets are
>> >>>> still
>> >>>>>> needed, then perhaps those project volunteers can contribute the
>> >>>> needed
>> >>>>>> updates to keep the functionality useful.
>> >>>>>>
>> >>>>>> Thanks
>> >>>>>> Thanks
>> >>>>>>
>> >>>>>>
>> >>>>>> On Mon, Aug 18, 2025 at 6:48 AM Sifiso Mtetwa <
>> >>>> sifiso@skyburgsystems.org>
>> >>>>>> wrote:
>> >>>>>>
>> >>>>>>> Hi guys,
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> This is an interesting topic. I have been wondering why, in
>> general,
>> >>>> we
>> >>>>>>> seem to be deprecating more and more system functions. The
>> individual
>> >>>>>>> collection sheet has served us well over the years and still
>> does, If
>> >>>>>>> anything we could improve on its functionality by maybe adding a
>> bulk
>> >>>>>>> collection sheet template with little detail compared to a full
>> loan
>> >>>>>>> repayment bulk import template.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Fineract is used by tens of thousands of organisations throughout
>> the
>> >>>>>>> world and most of them are not on this listing and may not have a
>> >>>> voice to
>> >>>>>>> air their concerns. Maybe we can find a way of exposing this
>> thread
>> >>>> to
>> >>>>>>> include more voters.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Regards,
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> *From:* James Dailey [mailto:jdailey@apache.org]
>> >>>>>>> *Sent:* Saturday, 16 August 2025 02:53
>> >>>>>>> *To:* dev@fineract.apache.org
>> >>>>>>> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
>> >>>>>>> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Ed
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> I agree this needs to be discussed but it is important to
>> acknowledge
>> >>>>>>> that THIS Fineract listserv is the only official (and required)
>> >>>> discussion
>> >>>>>>> space.  It is not the intention to bury anything in “an email
>> >>>> thread”.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> When I started up Mifos we spent a lot of time looking at
>> Collection
>> >>>>>>> Sheets and designing process flows around them. I fully know that
>> >>>> this
>> >>>>>>> design direction was important back then in 2002-2006.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> However if there is no one here asking for them to be retained
>> >>>> besides
>> >>>>>>> you, then that is a sign that they have perhaps reached an end of
>> >>>> their
>> >>>>>>> utility.  Or, that the users are not actually here, which is a
>> >>>> different
>> >>>>>>> problem.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> So,… If there is a group using collection sheets in production AND
>> >>>> they
>> >>>>>>> are not on some permanent forked (old) version, then now is the
>> time
>> >>>> to
>> >>>>>>> speak up.  Here.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Generally, I’m fairly certain we can refactor this with an eye
>> toward
>> >>>>>>> extracting it from the core.  Repeating the logic in two places
>> >>>> makes no
>> >>>>>>> sense either.  Collection sheets are kind of assembled from
>> >>>>>>> constituent loans and savings.  The balances and due payments
>> should
>> >>>> be
>> >>>>>>> calculated in the underlying components.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> As the system gets restructured we need to decide to keep this at
>> >>>> all, to
>> >>>>>>> keep it in a new place, or as some external concept/plug in. Why
>> >>>> wouldn’t
>> >>>>>>> we want a separate component?
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Cheers
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Thu, Aug 14, 2025 at 4:16 PM Ed Cable <ed...@mifos.org>
>> wrote:
>> >>>>>>>
>> >>>>>>> I'll leave the other thread for discussion of the API versioning
>> and
>> >>>>>>> refactoring related to Collection Sheet API but wanted to create a
>> >>>> separate
>> >>>>>>> thread regarding the deprecation of the Collection Sheet.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> In general, for this and removal of any functionality, it's
>> something
>> >>>>>>> that needs to be discussed openly with the community and with a
>> vote
>> >>>> and
>> >>>>>>> not a decision buried in a mailing list thread.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> For the collection sheet specifically, more thought has to be
>> given
>> >>>> to
>> >>>>>>> its deprecation as the centrality and highly coupled nature of the
>> >>>>>>> collection sheet is being understated as it isn't merely a report
>> >>>> that's to
>> >>>>>>> be printed or a form filled out via a mobile application. It's a
>> >>>>>>> significant portion of the user interface and highly coupled to
>> many
>> >>>> of the
>> >>>>>>> microfinance features around groups/centers/meeting scheduling,
>> staff
>> >>>>>>> assignment, etc.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> I do agree that from a UI perspective, the collection sheet and
>> other
>> >>>>>>> microfinance-centric functionalities and flows should be viewable
>> >>>> based on
>> >>>>>>> a configurable setting. As it doesn't lend itself to the optimal
>> user
>> >>>>>>> experience for a large portion of current Fineract user base.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> I also am supportive of a strategy of slimming Fineract to its
>> core
>> >>>>>>> services and functionality above core Fineract services and APIs
>> can
>> >>>> be
>> >>>>>>> extracted out.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> So I do think we should give thoughtful consideration to what
>> >>>> abstracting
>> >>>>>>> out the collection sheet and corresponding microfinance
>> >>>> functionality would
>> >>>>>>> look like and what that effort would entail to abstract it out
>> >>>> without
>> >>>>>>> adversely impacting the original user base of the software but
>> it's
>> >>>> not as
>> >>>>>>> simple as deprecating these API.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> I welcome others' thoughts and inputs as I know even with
>> >>>> microfinance
>> >>>>>>> itself, the methodology has evolved and group lending and the
>> >>>> concept of a
>> >>>>>>> collection sheet isn't as central as it once was
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Thanks,
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Ed
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Wed, Aug 13, 2025 at 6:36 PM Kapil Panchal <
>> >>>>>>> kapil.panchal.development@gmail.com> wrote:
>> >>>>>>>
>> >>>>>>> Hi James,
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> I can proceed by marking this feature as @Deprecated and/or
>> >>>> performing a
>> >>>>>>> safe refactor to remove the API.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> For API versioning I agree to Aleksandar Vidakovic's proposal on
>> >>>> adapting
>> >>>>>>> to the SpringBoot v7/Spring Framework v4. If I may, Aleksandar
>> >>>>>>> Vidakovic takes the lead on this project and I can help to support
>> >>>> the
>> >>>>>>> conversion?
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Thanks,
>> >>>>>>>
>> >>>>>>> Kapil
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Thu, Aug 14, 2025 at 5:07 AM James Dailey <jd...@apache.org>
>> >>>> wrote:
>> >>>>>>>
>> >>>>>>> Hi Kapil
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> I might suggest looking at this as an opportunity to remove the
>> >>>>>>> collection sheet entirely from the Fineract namespace.  It’s a
>> legacy
>> >>>>>>> concept I and others designed a long time ago, originally in 2002
>> >>>> based on
>> >>>>>>> collection sheets we gathered from a dozen countries. It is
>> strongly
>> >>>> tied
>> >>>>>>> to concepts in microfinance field operations, and especially when
>> >>>> there was
>> >>>>>>> no data connectivity.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> It belongs perhaps as a sort of external microservice - data
>> loading
>> >>>> via
>> >>>>>>> a bulk import could still be enabled.
>> >>>>>>>
>> >>>>>>> The API versioning is a good idea but needs to be more holistic
>> >>>> across
>> >>>>>>> the platform I think.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <ad...@gmail.com>
>> >>>> wrote:
>> >>>>>>>
>> >>>>>>> Hi Kapil,
>> >>>>>>>
>> >>>>>>> Thank you for raising the concerns below. I’ll need some
>> additional
>> >>>>>>> details to fully understand your points:
>> >>>>>>>
>> >>>>>>> 1.      *Collection Sheet API* – You mentioned it appears
>> >>>> non-functional
>> >>>>>>> and contains several logical errors.
>> >>>>>>>
>> >>>>>>> o    If it’s indeed not working, that’s a separate, high-priority
>> >>>>>>> discussion.
>> >>>>>>>
>> >>>>>>> o    Could you clarify which logical errors you were referring to,
>> >>>> and
>> >>>>>>> what specifically makes you think it’s non-functional?
>> >>>>>>>
>> >>>>>>> 2.      *Service annotations* – You noted that service methods are
>> >>>> not
>> >>>>>>> annotated with @Service and that beans are defined manually.
>> >>>>>>>
>> >>>>>>> o    Are you referring to the
>> >>>>>>> CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being
>> >>>> defined
>> >>>>>>> via configuration?
>> >>>>>>>
>> >>>>>>> 3.      *Repository wrappers annotated with **@Service* – You
>> >>>> mentioned
>> >>>>>>> that this mandates full unit test coverage but that they should
>> >>>> ideally be
>> >>>>>>> annotated with @Component.
>> >>>>>>>
>> >>>>>>> o    Could you point out the exact classes you had in mind?
>> >>>>>>>
>> >>>>>>> As for the other points, I agree we can refactor and remove
>> redundant
>> >>>>>>> logic—please feel free to suggest specific improvements or start
>> >>>> work on
>> >>>>>>> them immediately!
>> >>>>>>>
>> >>>>>>> However, be careful by moving anything into the fineract-core… We
>> are
>> >>>>>>> aiming to keep it as small as possible as everything is built on
>> top
>> >>>> of
>> >>>>>>> this module! If collection sheet are used for loans and savings -
>> for
>> >>>>>>> example - than the recommended move is NOT to move this logic into
>> >>>> core!
>> >>>>>>>
>> >>>>>>> Either:
>> >>>>>>>
>> >>>>>>> - we split the logic into fineract-loan and fineract-savings
>> >>>>>>>
>> >>>>>>> - Move the logic into a new module
>> >>>>>>>
>> >>>>>>> - Leave it in fineract-provider for now
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Shall you have any questions, please let us know!
>> >>>>>>>
>> >>>>>>> Regards,
>> >>>>>>> Adam
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On 2025. Aug 11., at 12:09, Kapil Panchal <
>> >>>>>>> kapil.panchal.development@gmail.com> wrote:
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> Hi Adam,
>> >>>>>>>
>> >>>>>>> I’m currently working on *FINERACT-2290* and have a few questions
>> >>>> before
>> >>>>>>> I submit a pull request.
>> >>>>>>>
>> >>>>>>> The *Collection Sheet API* in its current state appears
>> >>>> non-functional
>> >>>>>>> and contains several logical errors. It seems there was an earlier
>> >>>> attempt
>> >>>>>>> to convert from a JSON string request parameter to a class-based
>> >>>> request
>> >>>>>>> object, but:
>> >>>>>>>
>> >>>>>>> Certain fields are missing.
>> >>>>>>>
>> >>>>>>> The serializer is not correctly populating the objects, which
>> causes
>> >>>> the
>> >>>>>>> conditional checks to be bypassed and results in incorrect (false)
>> >>>>>>> responses.
>> >>>>>>>
>> >>>>>>> This change set is *high risk* because it touches most of the loan
>> >>>> and
>> >>>>>>> savings product logic. I’ve had to refactor almost all major
>> methods.
>> >>>>>>> Extensive integration and end-to-end testing will be required to
>> >>>> ensure
>> >>>>>>> there are no regressions, especially in edge cases. At present,
>> >>>> there are
>> >>>>>>> no unit or integration tests for this functionality, and test
>> >>>> creation is
>> >>>>>>> outside the current ticket scope. I’ve been iterating on this for
>> a
>> >>>> while,
>> >>>>>>> and only today have I reached a stable state after several
>> >>>> experimental and
>> >>>>>>> build-breaking attempts.
>> >>>>>>>
>> >>>>>>> *Key Observations:*
>> >>>>>>>
>> >>>>>>> Service methods are not annotated with @Service; instead, beans
>> are
>> >>>>>>> defined manually.
>> >>>>>>>
>> >>>>>>> Repository wrappers are annotated with @Service. This mandates
>> full
>> >>>> unit
>> >>>>>>> test coverage for these methods, but they should ideally be
>> >>>> annotated with
>> >>>>>>> @Component.
>> >>>>>>>
>> >>>>>>> I agree with prior discussions on separating bean validation —
>> >>>> having a
>> >>>>>>> dedicated @Component validation class allows the request object to
>> >>>> handle
>> >>>>>>> checks independent of database queries.
>> >>>>>>>
>> >>>>>>> Validation components can also perform database-related
>> validations;
>> >>>>>>> these can be injected into service classes for cleaner
>> architecture.
>> >>>>>>>
>> >>>>>>> Such validation components should be placed in *Fineract-Core* so
>> >>>> they
>> >>>>>>> are reusable across modules, reducing future refactoring needs.
>> >>>>>>>
>> >>>>>>> The current design of having commands in *Fineract-Core* and
>> >>>>>>> handlers/services/repositories in respective modules is good — it
>> >>>> cleanly
>> >>>>>>> decouples command definition from execution.
>> >>>>>>>
>> >>>>>>> There is extensive use of this. in singleton contexts (API,
>> Service,
>> >>>>>>> Repository). While not harmful, it’s unnecessary boilerplate.
>> >>>>>>>
>> >>>>>>> Multiple redundant intermediate DTOs exist where the request DTO
>> >>>> itself
>> >>>>>>> could be reused for data transfer.
>> >>>>>>>
>> >>>>>>> I found redundant logic — e.g., a for loop with a break statement
>> >>>> that
>> >>>>>>> effectively executes only once; this can be simplified.
>> >>>>>>>
>> >>>>>>> Some JDBC template queries use reserved SQL keywords, causing
>> >>>> exceptions.
>> >>>>>>> Refactoring these queries resolves the issue and returns proper
>> >>>> response
>> >>>>>>> objects.
>> >>>>>>>
>> >>>>>>> *Suggestions:*
>> >>>>>>>
>> >>>>>>> *Where appropriate, large tickets should be broken into subtasks
>> to
>> >>>>>>> manage complexity and reviewability.*
>> >>>>>>>
>> >>>>>>> It may help to have a dedicated *developer-only Slack channel *for
>> >>>>>>> technical discussions. This could complement other community
>> spaces
>> >>>> if
>> >>>>>>> there’s a need to keep certain conversations more focused.
>> >>>>>>>
>> >>>>>>> What are your thoughts on the above?
>> >>>>>>>
>> >>>>>>> Thanks,
>> >>>>>>> Kapil
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>>
>> >>>>>>> *Ed Cable*
>> >>>>>>>
>> >>>>>>> President/CEO, Mifos Initiative
>> >>>>>>>
>> >>>>>>> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> *Collectively Creating a World of 3 Billion Maries | *
>> >>>> http://mifos.org
>> >>>>>>> <http://facebook.com/mifos>  <http://www.twitter.com/mifos>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >
>>
>>
>
> --
> --
> Paul
>


-- 
*Ed Cable*
President/CEO, Mifos Initiative
edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649

*Collectively Creating a World of 3 Billion Maries | *http://mifos.org
<http://facebook.com/mifos>  <http://www.twitter.com/mifos>

Re: Mission of Fineract discussion (was Re: Collection Sheet Deprecation ...)

[Paul <pc...@gmail.com>]

Hi Felix,
My POV was directed for the whole community . . . and was not a
"counterpoint" to anyone.
Maybe an extension of thought and method? (why the broader Fineract mission
is good for unbanked and a "how to" example.).



On Sun, Aug 31, 2025 at 7:41 AM Felix van Hove <fv...@gmx.de.invalid>
wrote:

> Paul, I've thought a long time on your email. I think I find it
> difficult to answer, because - to a certain extend - I agree with you
> and James. It might be good to position Fineract broadly like this.
>
> But a discussion on a roadmap, with all parties involved, is of
> importance. To ask on the mailing list "Who is willing to fix a buggy
> feature X? Who will support it in the future?" and decommission it
> afterwards, because no one showed up, - appears too simple to me.
>
> Though I think I had more confidence in this happening, if I saw *any*
> new feature being proposed during my time, serving the same people that
> the Collection Sheet serves.
>
> Felix
>
>
>
> On 30/08/2025 22:49, Paul wrote:
> > Great feedback, Felix, Zayyad.
> > I would like to share a perspective from history and why the big picture
> > approach and broader mission is so important.
> >
> > There was a phrase in use when I was a baby. "Separate, but Equal." This
> > was a phrase used to justify keeping US school systems segregated by
> race.
> > "Separate but Equal" was a lie. The systems were never equal in funding,
> > quality or anything else.
> > Finance and access is the same.
> >
> > *What does that have to do with Fineract?*
> > Mission solely dedicated to the unbanked shrinks the number of people
> > willing to join the community.
> > Others may be sympathetic, but won't directly participate by
> contributing.
> > I happen to be interested in the unbanked and personally support that
> > mission.
> >
> > HOWEVER, my main interest in Fineract is to support a commercial
> enterprise
> > Open Source core banking system. A core banking system which integrates
> > embedds services anywhere, not just microfinance and money transfers.
> >
> > When the Fineract mission was focused on the unbanked, it was seriously
> > under participated IMO. With its slight change in mission which is still
> > inclusive of the underbanked, participation is increasing and releases
> are
> > beginning to become more regular. The unbanked needs to be supported
> with a
> > system capable of full integration with the rest of the world, not a
> > "separate but equal system".
> >
> > I hope my history story makes sense to at least support my POV.
> >
> > *How Fineract works for the unbanked with its current mission
> statement.  *
> >
> > MIFOS is solely dedicated to the underserved and has the frontend to do
> so.
> > In my opinion, if collection sheets are still used in working with
> > unbanked, then MIFOS community should adopt and help maintain them them
> as
> > a current, modern, secure function so that the MIFOS frontend works with
> > the Fineract backend. MIFOS like all other users of Fineract get the
> > benefit of a stronger core banking system and vendors with dedicated
> > purpose work together to maintain their particular core channels within
> > Fineract.
> >
> > The image is a generalization to convey the concept. (It may not be
> precise
> > as to the vendors and players, but should reinforce my POV.)
> > So, you, your interest or company wants a function or to maintain an
> > existing function? GREAT!
> > Step up! Own that solution within Fineract.
> > [image: image.png]
> > I speak solely for myself and my POV, but this approach will generate 10X
> > deeper support than segregated methods.
> >
> > In the past, there was insufficient community, but I think there was also
> > "community orchestration" missing to a degree.
> >
> > *Fail Model:*
> > Hoping it all gets done and telling the community at large "do something"
> > doesn't work.
> >
> > *Success Model: *
> > When a platform is useful across a whole ecosystem, then it can be
> jointly
> > maintained with nominal management. People are willing to help. Just like
> > musicians want to play music, they want to work together. In RARE
> > circumstances, the community is so SINGLE minded, the build, like JAZZ,
> > happens spontaneously and all agree, its really cool.
> >
> > *However, the bulk of the world needs direction and orchestration* to
> "make
> > music" of product delivery. Because of those many different interests,
> > Fineract needs process to harmonize and create results.
> >
> > In the end, there also must be a binary decision, "yes or no" to support
> an
> > issue. It comes down to; " will a community member support a function
> > important to their channel?"
> >
> > I'm not a developer, my useful skills are domain knowledge and
> > instructional orchestration (process). So I challenge all community
> members
> > to find a channel required to fulfill your channel or interest and
> > volunteer to "own it" through a date certain. Better yet, form a small
> team
> > of like minds and assure that function is ALWAYS current and secure.
> >
> > This is a challenge to all community members and targeting no one.
> >
> > Paul
> >
> > On Sat, Aug 30, 2025 at 3:34 PM James Dailey <jd...@apache.org> wrote:
> >
> >> Zayyad, Felix and others -
> >>
> >> Indeed.  And all of this was laid out for many years as the strategy.
> >>
> >> When I was on the Board of Mifos (the non profit, not the project) we
> >> decided to contribute the MifosX code to the ASF to ensure both:
> >>    a) a broader financial services industry approach, and
> >>    b) a more established foundational home for the core of the project.
> >>
> >> There were other reasons, but those remain primary in my mind.
> >>
> >> The concept was that Mifos org, which is a Vendor in the ASF parlance,
> >> would continue its Financial Inclusion mission and make sure that the
> >> releases that were coming out of the project would retain the flavor and
> >> intent of the original mifos project (which I started and designed in
> >> 2001).  The Mifos AdminUI /CommunityApp open source front end UI gave
> it a
> >> place in the ecosystem and a way to ensure that alignment.
> >>
> >> But, importantly, when we made this decision, we deliberately wanted to
> >> see more financial institutions and fintechs use the backend system, AND
> >> vitally, to help maintain it, with contributions upstream.  That is the
> >> sign of a healthy open source project.
> >>
> >> I don't speak for Mifos, but I believe that the vision at Mifos remains
> >> about financial inclusion.   But the vision of Fineract and the mission
> of
> >> the project is different.  We actually VOTED on this revised statement
> as a
> >> community, I think five years ago.
> >>
> >> So, if you are seeing a difference between Mifos and Fineract, then we
> >> have succeeded in an important goal.  The two are NOT THE SAME.  This is
> >> NOT mifos.
> >>
> >> This has been called out in many many Reports to the ASF Board.  See the
> >> wiki pages for those.  The key theme is Vendor neutrality - we do not
> >> privilege any vendor.
> >>
> >> And, I would argue that this is the right healthy direction for this
> >> Fineract project.  Microfinance serves an important role in the world,
> and
> >> continues despite many expecting it to decline or go away.  And, the
> >> mission of the original concept of Mifos was not about Microfinance but
> >> about financial inclusion.  However that happens.
> >>
> >> And, fundamentally, I personally remain focused on this - disrupting the
> >> financial industry with open source software that can functionally be
> the
> >> basis of any bank or fintech globally.  It should lower the cost of
> >> financial inclusion, it should do that via any kind of institution at a
> >> structural level.  Unbanked and underbanked are included.
> >>
> >> So, if you are not seeing what you want here, then you can invest more
> >> time and resources to make it more of what you want, but I would also
> >> suggest that the fundamental thing is to have healthy vendors that move
> the
> >> project forward with lots of "built on top of" functionality that serve
> a
> >> wide range of financial institutions.  If those components are also open
> >> source, that's ok, but it isn't essential.
> >>
> >> ** Deprecation process **
> >> Deprecating features is a natural part of any system evolution - we need
> >> to prune the tree from time to time.  If users are demanding those
> features
> >> then they should speak up - as this thread illustrated - and mechanisms
> >> need to be put in place to "protect them".  Here of course, I am
> referring
> >> to my last email and testing, to start with.
> >>
> >> And, we should do a better job of documenting the product roadmap, and
> >> having clear discussions about deprecating features - Before they are
> >> deprecated.
> >>
> >> That starts with you listing your required feature set and documenting
> it
> >> somewhere perhaps - or relying on the Vendor to speak up for your
> interests
> >> if you don't.
> >>
> >> And, this is a good discussion to have.  Perhaps a community call on
> this
> >> topic would be welcomed in coming weeks.
> >>
> >> James
> >>
> >>
> >>
> >> On Sat, Aug 30, 2025 at 3:13 AM Zayyad A. Said <
> >> zayyad@intrasofttechnologies.com> wrote:
> >>
> >>> Hello Fineract community,
> >>>
> >>> I concur with the observations below from Felix.
> >>>
> >>> We have been implementing Mifos since 2011 and we have seen the project
> >>> transitioning from Mifos 2.x to Mifos X and now with the new
> developments.
> >>> The current focus seems to have shifted from the original mission of
> >>> "Creating a world of 3 Billion Maries where each of the 2 billion poor
> and
> >>> unbanked has access to the financial resources needed to create a
> better
> >>> life for themselves and their family."
> >>>
> >>> Group lending has been used for decades and Mifos was originally
> intended
> >>> for and served the model, the new features being introduced are geared
> >>> towards commercial use. We saw this coming and we had to maintain old
> fork
> >>> for our MFI clients where we continue to improve based on their needs.
> >>>
> >>> While I have no problem with having new features, I seem not to
> >>> understand why we want to deprecate features of group lending that are
> >>> still being used even if with the minority of users globally yet they
> do no
> >>> harm to the security and overall performance of the project.
> >>>
> >>> I suggest the leadership to rethink the idea of deprecating collection
> >>> sheet functionality and other key features used in group lending.
> >>>
> >>> Best Regards,
> >>>
> >>> Zayyad A. Said
> >>> Intrasoft Technologies Limited
> >>>
> >>>   *********
> >>>
> >>> *Zayyad A. Said | Chief Executive Officer*
> >>>
> >>> Suite H32, Delamere Flats - Milimani Road, Nairobi.
> >>>
> >>> Cell No.: +254 716 615274 | Skype: *zsaid2011*
> >>>
> >>> Email: zayyad@intrasofttechnologies.com
> >>>
> >>> Schedule Meetings: https://calendly.com/zayyadsaid
> >>>
> >>> <https://calendly.com/zayyadsaid>
> >>>
> >>>
> >>> *" You can achieve what you want if you just help enough others get
> what
> >>> they want.."*
> >>>
> >>> *Note: Save the environment, please don't print this email unless its
> >>> really necessary for you to do that.*
> >>>
> >>> On Sat, Aug 30, 2025, 11:20 Felix van Hove <fv...@gmx.de.invalid>
> >>> wrote:
> >>>
> >>>> Let me chip in with a more personal perspective.
> >>>>
> >>>> I'm following the development of the Mifos web app and - to a certain
> >>>> extend - Fineract for a couple of months only. My impression from the
> >>>> influx of new features in Fineract is that these are predominantly not
> >>>> for microfinance or small entities, which Mifos once set out to
> support.
> >>>> In fact - Fineract only states that its efforts are also "including
> the
> >>>> unbanked and underbanked" - not its focus.
> >>>>
> >>>> I see the deprecation of the Collection Sheet feature in this context.
> >>>> How many new features have been added in the last 12 months that help
> >>>> the "unbanked"? How many have been added that help larger commercial
> >>>> entities and are of no benefit for "the unbanked"?
> >>>>
> >>>> My personal contributions depend solely on Mifos/Fineract supporting
> >>>> efforts for financial inclusion. I regard it as of utter importance
> >>>> respective features don't come as a plugin, but first class citizens.
> >>>>
> >>>> My knowledge regarding fintech is very limited. If I'm wrong or
> >>>> misrespresent things, please tell me.
> >>>>
> >>>> Felix
> >>>>
> >>>> On 29/08/2025 21:01, James Dailey wrote:
> >>>>> Devs -  Are we done with this?
> >>>>>
> >>>>> First, I am NOT ruling out this functionality as important but if no
> >>>> one is
> >>>>> actually demanding it, and no one is willing to maintain it, then it
> >>>>> doesn't get maintained.  That is the nature of open source projects.
> >>>>>
> >>>>> Second, it can happen that an outside vendor (e.g. Mifos) that relies
> >>>>> on this Collection Sheet functionality (or any functionality)
> >>>> inadvertently
> >>>>> drops that from their front end release.  Because the tests at
> >>>> Fineract do
> >>>>> not cover this fully, no one at the Vendor and no one at the Fineract
> >>>>> project will see the functionality fail in a build until a customer
> or
> >>>>> implementer notices. So, tests are vital for ensuring ongoing
> >>>>> maintainability.
> >>>>>
> >>>>> Third, if this type of thing isn't "naturally" part of the core -
> then
> >>>> it
> >>>>> will make a lot more sense to have it be an outside extension - in
> >>>> which
> >>>>> case the tests have to be written for the API calls, and a Vendor
> >>>> should
> >>>>> try to get such tests contributed.   I could be wrong about this, I
> >>>> would
> >>>>> be happy to debate where this belongs.
> >>>>>
> >>>>> @Edward Cable <ed...@mifos.org> - what is the assessment and
> status
> >>>> of
> >>>>> needed functionality and where do you think it should live?
> >>>>>
> >>>>> Thanks,
> >>>>> James
> >>>>>
> >>>>> On Mon, Aug 18, 2025 at 8:21 AM James Dailey <jd...@apache.org>
> >>>> wrote:
> >>>>>
> >>>>>> Sifiso -
> >>>>>>
> >>>>>> Do we have a way to reach the tens of thousands of institutions you
> >>>>>> believe are there?
> >>>>>>
> >>>>>> My belief (just anecdotal information) is that it’s more like a few
> >>>>>> hundred institutions and that out of that number, less than 10% are
> >>>> on a
> >>>>>> recent release.  And, of that smaller number, few, if any, are still
> >>>> using
> >>>>>> Collection Sheets.  I could be wrong - I’d like the data!  I’d say
> >>>> that
> >>>>>> vendors can show up with their data here to help make this vendor
> >>>> neutral
> >>>>>> space more informed.
> >>>>>>
> >>>>>> If group lending and group-member lending and collection sheets are
> >>>> still
> >>>>>> needed, then perhaps those project volunteers can contribute the
> >>>> needed
> >>>>>> updates to keep the functionality useful.
> >>>>>>
> >>>>>> Thanks
> >>>>>> Thanks
> >>>>>>
> >>>>>>
> >>>>>> On Mon, Aug 18, 2025 at 6:48 AM Sifiso Mtetwa <
> >>>> sifiso@skyburgsystems.org>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hi guys,
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> This is an interesting topic. I have been wondering why, in
> general,
> >>>> we
> >>>>>>> seem to be deprecating more and more system functions. The
> individual
> >>>>>>> collection sheet has served us well over the years and still does,
> If
> >>>>>>> anything we could improve on its functionality by maybe adding a
> bulk
> >>>>>>> collection sheet template with little detail compared to a full
> loan
> >>>>>>> repayment bulk import template.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Fineract is used by tens of thousands of organisations throughout
> the
> >>>>>>> world and most of them are not on this listing and may not have a
> >>>> voice to
> >>>>>>> air their concerns. Maybe we can find a way of exposing this thread
> >>>> to
> >>>>>>> include more voters.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Regards,
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> *From:* James Dailey [mailto:jdailey@apache.org]
> >>>>>>> *Sent:* Saturday, 16 August 2025 02:53
> >>>>>>> *To:* dev@fineract.apache.org
> >>>>>>> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
> >>>>>>> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Ed
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> I agree this needs to be discussed but it is important to
> acknowledge
> >>>>>>> that THIS Fineract listserv is the only official (and required)
> >>>> discussion
> >>>>>>> space.  It is not the intention to bury anything in “an email
> >>>> thread”.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> When I started up Mifos we spent a lot of time looking at
> Collection
> >>>>>>> Sheets and designing process flows around them. I fully know that
> >>>> this
> >>>>>>> design direction was important back then in 2002-2006.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> However if there is no one here asking for them to be retained
> >>>> besides
> >>>>>>> you, then that is a sign that they have perhaps reached an end of
> >>>> their
> >>>>>>> utility.  Or, that the users are not actually here, which is a
> >>>> different
> >>>>>>> problem.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> So,… If there is a group using collection sheets in production AND
> >>>> they
> >>>>>>> are not on some permanent forked (old) version, then now is the
> time
> >>>> to
> >>>>>>> speak up.  Here.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Generally, I’m fairly certain we can refactor this with an eye
> toward
> >>>>>>> extracting it from the core.  Repeating the logic in two places
> >>>> makes no
> >>>>>>> sense either.  Collection sheets are kind of assembled from
> >>>>>>> constituent loans and savings.  The balances and due payments
> should
> >>>> be
> >>>>>>> calculated in the underlying components.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> As the system gets restructured we need to decide to keep this at
> >>>> all, to
> >>>>>>> keep it in a new place, or as some external concept/plug in. Why
> >>>> wouldn’t
> >>>>>>> we want a separate component?
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Cheers
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Thu, Aug 14, 2025 at 4:16 PM Ed Cable <ed...@mifos.org>
> wrote:
> >>>>>>>
> >>>>>>> I'll leave the other thread for discussion of the API versioning
> and
> >>>>>>> refactoring related to Collection Sheet API but wanted to create a
> >>>> separate
> >>>>>>> thread regarding the deprecation of the Collection Sheet.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> In general, for this and removal of any functionality, it's
> something
> >>>>>>> that needs to be discussed openly with the community and with a
> vote
> >>>> and
> >>>>>>> not a decision buried in a mailing list thread.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> For the collection sheet specifically, more thought has to be given
> >>>> to
> >>>>>>> its deprecation as the centrality and highly coupled nature of the
> >>>>>>> collection sheet is being understated as it isn't merely a report
> >>>> that's to
> >>>>>>> be printed or a form filled out via a mobile application. It's a
> >>>>>>> significant portion of the user interface and highly coupled to
> many
> >>>> of the
> >>>>>>> microfinance features around groups/centers/meeting scheduling,
> staff
> >>>>>>> assignment, etc.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> I do agree that from a UI perspective, the collection sheet and
> other
> >>>>>>> microfinance-centric functionalities and flows should be viewable
> >>>> based on
> >>>>>>> a configurable setting. As it doesn't lend itself to the optimal
> user
> >>>>>>> experience for a large portion of current Fineract user base.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> I also am supportive of a strategy of slimming Fineract to its core
> >>>>>>> services and functionality above core Fineract services and APIs
> can
> >>>> be
> >>>>>>> extracted out.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> So I do think we should give thoughtful consideration to what
> >>>> abstracting
> >>>>>>> out the collection sheet and corresponding microfinance
> >>>> functionality would
> >>>>>>> look like and what that effort would entail to abstract it out
> >>>> without
> >>>>>>> adversely impacting the original user base of the software but it's
> >>>> not as
> >>>>>>> simple as deprecating these API.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> I welcome others' thoughts and inputs as I know even with
> >>>> microfinance
> >>>>>>> itself, the methodology has evolved and group lending and the
> >>>> concept of a
> >>>>>>> collection sheet isn't as central as it once was
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Ed
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Aug 13, 2025 at 6:36 PM Kapil Panchal <
> >>>>>>> kapil.panchal.development@gmail.com> wrote:
> >>>>>>>
> >>>>>>> Hi James,
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> I can proceed by marking this feature as @Deprecated and/or
> >>>> performing a
> >>>>>>> safe refactor to remove the API.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> For API versioning I agree to Aleksandar Vidakovic's proposal on
> >>>> adapting
> >>>>>>> to the SpringBoot v7/Spring Framework v4. If I may, Aleksandar
> >>>>>>> Vidakovic takes the lead on this project and I can help to support
> >>>> the
> >>>>>>> conversion?
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>>
> >>>>>>> Kapil
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Thu, Aug 14, 2025 at 5:07 AM James Dailey <jd...@apache.org>
> >>>> wrote:
> >>>>>>>
> >>>>>>> Hi Kapil
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> I might suggest looking at this as an opportunity to remove the
> >>>>>>> collection sheet entirely from the Fineract namespace.  It’s a
> legacy
> >>>>>>> concept I and others designed a long time ago, originally in 2002
> >>>> based on
> >>>>>>> collection sheets we gathered from a dozen countries. It is
> strongly
> >>>> tied
> >>>>>>> to concepts in microfinance field operations, and especially when
> >>>> there was
> >>>>>>> no data connectivity.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> It belongs perhaps as a sort of external microservice - data
> loading
> >>>> via
> >>>>>>> a bulk import could still be enabled.
> >>>>>>>
> >>>>>>> The API versioning is a good idea but needs to be more holistic
> >>>> across
> >>>>>>> the platform I think.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <ad...@gmail.com>
> >>>> wrote:
> >>>>>>>
> >>>>>>> Hi Kapil,
> >>>>>>>
> >>>>>>> Thank you for raising the concerns below. I’ll need some additional
> >>>>>>> details to fully understand your points:
> >>>>>>>
> >>>>>>> 1.      *Collection Sheet API* – You mentioned it appears
> >>>> non-functional
> >>>>>>> and contains several logical errors.
> >>>>>>>
> >>>>>>> o    If it’s indeed not working, that’s a separate, high-priority
> >>>>>>> discussion.
> >>>>>>>
> >>>>>>> o    Could you clarify which logical errors you were referring to,
> >>>> and
> >>>>>>> what specifically makes you think it’s non-functional?
> >>>>>>>
> >>>>>>> 2.      *Service annotations* – You noted that service methods are
> >>>> not
> >>>>>>> annotated with @Service and that beans are defined manually.
> >>>>>>>
> >>>>>>> o    Are you referring to the
> >>>>>>> CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being
> >>>> defined
> >>>>>>> via configuration?
> >>>>>>>
> >>>>>>> 3.      *Repository wrappers annotated with **@Service* – You
> >>>> mentioned
> >>>>>>> that this mandates full unit test coverage but that they should
> >>>> ideally be
> >>>>>>> annotated with @Component.
> >>>>>>>
> >>>>>>> o    Could you point out the exact classes you had in mind?
> >>>>>>>
> >>>>>>> As for the other points, I agree we can refactor and remove
> redundant
> >>>>>>> logic—please feel free to suggest specific improvements or start
> >>>> work on
> >>>>>>> them immediately!
> >>>>>>>
> >>>>>>> However, be careful by moving anything into the fineract-core… We
> are
> >>>>>>> aiming to keep it as small as possible as everything is built on
> top
> >>>> of
> >>>>>>> this module! If collection sheet are used for loans and savings -
> for
> >>>>>>> example - than the recommended move is NOT to move this logic into
> >>>> core!
> >>>>>>>
> >>>>>>> Either:
> >>>>>>>
> >>>>>>> - we split the logic into fineract-loan and fineract-savings
> >>>>>>>
> >>>>>>> - Move the logic into a new module
> >>>>>>>
> >>>>>>> - Leave it in fineract-provider for now
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Shall you have any questions, please let us know!
> >>>>>>>
> >>>>>>> Regards,
> >>>>>>> Adam
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On 2025. Aug 11., at 12:09, Kapil Panchal <
> >>>>>>> kapil.panchal.development@gmail.com> wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Hi Adam,
> >>>>>>>
> >>>>>>> I’m currently working on *FINERACT-2290* and have a few questions
> >>>> before
> >>>>>>> I submit a pull request.
> >>>>>>>
> >>>>>>> The *Collection Sheet API* in its current state appears
> >>>> non-functional
> >>>>>>> and contains several logical errors. It seems there was an earlier
> >>>> attempt
> >>>>>>> to convert from a JSON string request parameter to a class-based
> >>>> request
> >>>>>>> object, but:
> >>>>>>>
> >>>>>>> Certain fields are missing.
> >>>>>>>
> >>>>>>> The serializer is not correctly populating the objects, which
> causes
> >>>> the
> >>>>>>> conditional checks to be bypassed and results in incorrect (false)
> >>>>>>> responses.
> >>>>>>>
> >>>>>>> This change set is *high risk* because it touches most of the loan
> >>>> and
> >>>>>>> savings product logic. I’ve had to refactor almost all major
> methods.
> >>>>>>> Extensive integration and end-to-end testing will be required to
> >>>> ensure
> >>>>>>> there are no regressions, especially in edge cases. At present,
> >>>> there are
> >>>>>>> no unit or integration tests for this functionality, and test
> >>>> creation is
> >>>>>>> outside the current ticket scope. I’ve been iterating on this for a
> >>>> while,
> >>>>>>> and only today have I reached a stable state after several
> >>>> experimental and
> >>>>>>> build-breaking attempts.
> >>>>>>>
> >>>>>>> *Key Observations:*
> >>>>>>>
> >>>>>>> Service methods are not annotated with @Service; instead, beans are
> >>>>>>> defined manually.
> >>>>>>>
> >>>>>>> Repository wrappers are annotated with @Service. This mandates full
> >>>> unit
> >>>>>>> test coverage for these methods, but they should ideally be
> >>>> annotated with
> >>>>>>> @Component.
> >>>>>>>
> >>>>>>> I agree with prior discussions on separating bean validation —
> >>>> having a
> >>>>>>> dedicated @Component validation class allows the request object to
> >>>> handle
> >>>>>>> checks independent of database queries.
> >>>>>>>
> >>>>>>> Validation components can also perform database-related
> validations;
> >>>>>>> these can be injected into service classes for cleaner
> architecture.
> >>>>>>>
> >>>>>>> Such validation components should be placed in *Fineract-Core* so
> >>>> they
> >>>>>>> are reusable across modules, reducing future refactoring needs.
> >>>>>>>
> >>>>>>> The current design of having commands in *Fineract-Core* and
> >>>>>>> handlers/services/repositories in respective modules is good — it
> >>>> cleanly
> >>>>>>> decouples command definition from execution.
> >>>>>>>
> >>>>>>> There is extensive use of this. in singleton contexts (API,
> Service,
> >>>>>>> Repository). While not harmful, it’s unnecessary boilerplate.
> >>>>>>>
> >>>>>>> Multiple redundant intermediate DTOs exist where the request DTO
> >>>> itself
> >>>>>>> could be reused for data transfer.
> >>>>>>>
> >>>>>>> I found redundant logic — e.g., a for loop with a break statement
> >>>> that
> >>>>>>> effectively executes only once; this can be simplified.
> >>>>>>>
> >>>>>>> Some JDBC template queries use reserved SQL keywords, causing
> >>>> exceptions.
> >>>>>>> Refactoring these queries resolves the issue and returns proper
> >>>> response
> >>>>>>> objects.
> >>>>>>>
> >>>>>>> *Suggestions:*
> >>>>>>>
> >>>>>>> *Where appropriate, large tickets should be broken into subtasks to
> >>>>>>> manage complexity and reviewability.*
> >>>>>>>
> >>>>>>> It may help to have a dedicated *developer-only Slack channel *for
> >>>>>>> technical discussions. This could complement other community spaces
> >>>> if
> >>>>>>> there’s a need to keep certain conversations more focused.
> >>>>>>>
> >>>>>>> What are your thoughts on the above?
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Kapil
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>>
> >>>>>>> *Ed Cable*
> >>>>>>>
> >>>>>>> President/CEO, Mifos Initiative
> >>>>>>>
> >>>>>>> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> *Collectively Creating a World of 3 Billion Maries | *
> >>>> http://mifos.org
> >>>>>>> <http://facebook.com/mifos>  <http://www.twitter.com/mifos>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>
> >>>>
> >>>>
> >
>
>

-- 
--
Paul

Re: Mission of Fineract discussion (was Re: Collection Sheet Deprecation ...)

[Felix van Hove <fv...@gmx.de.INVALID>]

Paul, I've thought a long time on your email. I think I find it 
difficult to answer, because - to a certain extend - I agree with you 
and James. It might be good to position Fineract broadly like this.

But a discussion on a roadmap, with all parties involved, is of 
importance. To ask on the mailing list "Who is willing to fix a buggy 
feature X? Who will support it in the future?" and decommission it 
afterwards, because no one showed up, - appears too simple to me.

Though I think I had more confidence in this happening, if I saw *any* 
new feature being proposed during my time, serving the same people that 
the Collection Sheet serves.

Felix



On 30/08/2025 22:49, Paul wrote:
> Great feedback, Felix, Zayyad.
> I would like to share a perspective from history and why the big picture
> approach and broader mission is so important.
> 
> There was a phrase in use when I was a baby. "Separate, but Equal." This
> was a phrase used to justify keeping US school systems segregated by race.
> "Separate but Equal" was a lie. The systems were never equal in funding,
> quality or anything else.
> Finance and access is the same.
> 
> *What does that have to do with Fineract?*
> Mission solely dedicated to the unbanked shrinks the number of people
> willing to join the community.
> Others may be sympathetic, but won't directly participate by contributing.
> I happen to be interested in the unbanked and personally support that
> mission.
> 
> HOWEVER, my main interest in Fineract is to support a commercial enterprise
> Open Source core banking system. A core banking system which integrates
> embedds services anywhere, not just microfinance and money transfers.
> 
> When the Fineract mission was focused on the unbanked, it was seriously
> under participated IMO. With its slight change in mission which is still
> inclusive of the underbanked, participation is increasing and releases are
> beginning to become more regular. The unbanked needs to be supported with a
> system capable of full integration with the rest of the world, not a
> "separate but equal system".
> 
> I hope my history story makes sense to at least support my POV.
> 
> *How Fineract works for the unbanked with its current mission statement.  *
> 
> MIFOS is solely dedicated to the underserved and has the frontend to do so.
> In my opinion, if collection sheets are still used in working with
> unbanked, then MIFOS community should adopt and help maintain them them as
> a current, modern, secure function so that the MIFOS frontend works with
> the Fineract backend. MIFOS like all other users of Fineract get the
> benefit of a stronger core banking system and vendors with dedicated
> purpose work together to maintain their particular core channels within
> Fineract.
> 
> The image is a generalization to convey the concept. (It may not be precise
> as to the vendors and players, but should reinforce my POV.)
> So, you, your interest or company wants a function or to maintain an
> existing function? GREAT!
> Step up! Own that solution within Fineract.
> [image: image.png]
> I speak solely for myself and my POV, but this approach will generate 10X
> deeper support than segregated methods.
> 
> In the past, there was insufficient community, but I think there was also
> "community orchestration" missing to a degree.
> 
> *Fail Model:*
> Hoping it all gets done and telling the community at large "do something"
> doesn't work.
> 
> *Success Model: *
> When a platform is useful across a whole ecosystem, then it can be jointly
> maintained with nominal management. People are willing to help. Just like
> musicians want to play music, they want to work together. In RARE
> circumstances, the community is so SINGLE minded, the build, like JAZZ,
> happens spontaneously and all agree, its really cool.
> 
> *However, the bulk of the world needs direction and orchestration* to "make
> music" of product delivery. Because of those many different interests,
> Fineract needs process to harmonize and create results.
> 
> In the end, there also must be a binary decision, "yes or no" to support an
> issue. It comes down to; " will a community member support a function
> important to their channel?"
> 
> I'm not a developer, my useful skills are domain knowledge and
> instructional orchestration (process). So I challenge all community members
> to find a channel required to fulfill your channel or interest and
> volunteer to "own it" through a date certain. Better yet, form a small team
> of like minds and assure that function is ALWAYS current and secure.
> 
> This is a challenge to all community members and targeting no one.
> 
> Paul
> 
> On Sat, Aug 30, 2025 at 3:34 PM James Dailey <jd...@apache.org> wrote:
> 
>> Zayyad, Felix and others -
>>
>> Indeed.  And all of this was laid out for many years as the strategy.
>>
>> When I was on the Board of Mifos (the non profit, not the project) we
>> decided to contribute the MifosX code to the ASF to ensure both:
>>    a) a broader financial services industry approach, and
>>    b) a more established foundational home for the core of the project.
>>
>> There were other reasons, but those remain primary in my mind.
>>
>> The concept was that Mifos org, which is a Vendor in the ASF parlance,
>> would continue its Financial Inclusion mission and make sure that the
>> releases that were coming out of the project would retain the flavor and
>> intent of the original mifos project (which I started and designed in
>> 2001).  The Mifos AdminUI /CommunityApp open source front end UI gave it a
>> place in the ecosystem and a way to ensure that alignment.
>>
>> But, importantly, when we made this decision, we deliberately wanted to
>> see more financial institutions and fintechs use the backend system, AND
>> vitally, to help maintain it, with contributions upstream.  That is the
>> sign of a healthy open source project.
>>
>> I don't speak for Mifos, but I believe that the vision at Mifos remains
>> about financial inclusion.   But the vision of Fineract and the mission of
>> the project is different.  We actually VOTED on this revised statement as a
>> community, I think five years ago.
>>
>> So, if you are seeing a difference between Mifos and Fineract, then we
>> have succeeded in an important goal.  The two are NOT THE SAME.  This is
>> NOT mifos.
>>
>> This has been called out in many many Reports to the ASF Board.  See the
>> wiki pages for those.  The key theme is Vendor neutrality - we do not
>> privilege any vendor.
>>
>> And, I would argue that this is the right healthy direction for this
>> Fineract project.  Microfinance serves an important role in the world, and
>> continues despite many expecting it to decline or go away.  And, the
>> mission of the original concept of Mifos was not about Microfinance but
>> about financial inclusion.  However that happens.
>>
>> And, fundamentally, I personally remain focused on this - disrupting the
>> financial industry with open source software that can functionally be the
>> basis of any bank or fintech globally.  It should lower the cost of
>> financial inclusion, it should do that via any kind of institution at a
>> structural level.  Unbanked and underbanked are included.
>>
>> So, if you are not seeing what you want here, then you can invest more
>> time and resources to make it more of what you want, but I would also
>> suggest that the fundamental thing is to have healthy vendors that move the
>> project forward with lots of "built on top of" functionality that serve a
>> wide range of financial institutions.  If those components are also open
>> source, that's ok, but it isn't essential.
>>
>> ** Deprecation process **
>> Deprecating features is a natural part of any system evolution - we need
>> to prune the tree from time to time.  If users are demanding those features
>> then they should speak up - as this thread illustrated - and mechanisms
>> need to be put in place to "protect them".  Here of course, I am referring
>> to my last email and testing, to start with.
>>
>> And, we should do a better job of documenting the product roadmap, and
>> having clear discussions about deprecating features - Before they are
>> deprecated.
>>
>> That starts with you listing your required feature set and documenting it
>> somewhere perhaps - or relying on the Vendor to speak up for your interests
>> if you don't.
>>
>> And, this is a good discussion to have.  Perhaps a community call on this
>> topic would be welcomed in coming weeks.
>>
>> James
>>
>>
>>
>> On Sat, Aug 30, 2025 at 3:13 AM Zayyad A. Said <
>> zayyad@intrasofttechnologies.com> wrote:
>>
>>> Hello Fineract community,
>>>
>>> I concur with the observations below from Felix.
>>>
>>> We have been implementing Mifos since 2011 and we have seen the project
>>> transitioning from Mifos 2.x to Mifos X and now with the new developments.
>>> The current focus seems to have shifted from the original mission of
>>> "Creating a world of 3 Billion Maries where each of the 2 billion poor and
>>> unbanked has access to the financial resources needed to create a better
>>> life for themselves and their family."
>>>
>>> Group lending has been used for decades and Mifos was originally intended
>>> for and served the model, the new features being introduced are geared
>>> towards commercial use. We saw this coming and we had to maintain old fork
>>> for our MFI clients where we continue to improve based on their needs.
>>>
>>> While I have no problem with having new features, I seem not to
>>> understand why we want to deprecate features of group lending that are
>>> still being used even if with the minority of users globally yet they do no
>>> harm to the security and overall performance of the project.
>>>
>>> I suggest the leadership to rethink the idea of deprecating collection
>>> sheet functionality and other key features used in group lending.
>>>
>>> Best Regards,
>>>
>>> Zayyad A. Said
>>> Intrasoft Technologies Limited
>>>
>>>   *********
>>>
>>> *Zayyad A. Said | Chief Executive Officer*
>>>
>>> Suite H32, Delamere Flats - Milimani Road, Nairobi.
>>>
>>> Cell No.: +254 716 615274 | Skype: *zsaid2011*
>>>
>>> Email: zayyad@intrasofttechnologies.com
>>>
>>> Schedule Meetings: https://calendly.com/zayyadsaid
>>>
>>> <https://calendly.com/zayyadsaid>
>>>
>>>
>>> *" You can achieve what you want if you just help enough others get what
>>> they want.."*
>>>
>>> *Note: Save the environment, please don't print this email unless its
>>> really necessary for you to do that.*
>>>
>>> On Sat, Aug 30, 2025, 11:20 Felix van Hove <fv...@gmx.de.invalid>
>>> wrote:
>>>
>>>> Let me chip in with a more personal perspective.
>>>>
>>>> I'm following the development of the Mifos web app and - to a certain
>>>> extend - Fineract for a couple of months only. My impression from the
>>>> influx of new features in Fineract is that these are predominantly not
>>>> for microfinance or small entities, which Mifos once set out to support.
>>>> In fact - Fineract only states that its efforts are also "including the
>>>> unbanked and underbanked" - not its focus.
>>>>
>>>> I see the deprecation of the Collection Sheet feature in this context.
>>>> How many new features have been added in the last 12 months that help
>>>> the "unbanked"? How many have been added that help larger commercial
>>>> entities and are of no benefit for "the unbanked"?
>>>>
>>>> My personal contributions depend solely on Mifos/Fineract supporting
>>>> efforts for financial inclusion. I regard it as of utter importance
>>>> respective features don't come as a plugin, but first class citizens.
>>>>
>>>> My knowledge regarding fintech is very limited. If I'm wrong or
>>>> misrespresent things, please tell me.
>>>>
>>>> Felix
>>>>
>>>> On 29/08/2025 21:01, James Dailey wrote:
>>>>> Devs -  Are we done with this?
>>>>>
>>>>> First, I am NOT ruling out this functionality as important but if no
>>>> one is
>>>>> actually demanding it, and no one is willing to maintain it, then it
>>>>> doesn't get maintained.  That is the nature of open source projects.
>>>>>
>>>>> Second, it can happen that an outside vendor (e.g. Mifos) that relies
>>>>> on this Collection Sheet functionality (or any functionality)
>>>> inadvertently
>>>>> drops that from their front end release.  Because the tests at
>>>> Fineract do
>>>>> not cover this fully, no one at the Vendor and no one at the Fineract
>>>>> project will see the functionality fail in a build until a customer or
>>>>> implementer notices. So, tests are vital for ensuring ongoing
>>>>> maintainability.
>>>>>
>>>>> Third, if this type of thing isn't "naturally" part of the core - then
>>>> it
>>>>> will make a lot more sense to have it be an outside extension - in
>>>> which
>>>>> case the tests have to be written for the API calls, and a Vendor
>>>> should
>>>>> try to get such tests contributed.   I could be wrong about this, I
>>>> would
>>>>> be happy to debate where this belongs.
>>>>>
>>>>> @Edward Cable <ed...@mifos.org> - what is the assessment and status
>>>> of
>>>>> needed functionality and where do you think it should live?
>>>>>
>>>>> Thanks,
>>>>> James
>>>>>
>>>>> On Mon, Aug 18, 2025 at 8:21 AM James Dailey <jd...@apache.org>
>>>> wrote:
>>>>>
>>>>>> Sifiso -
>>>>>>
>>>>>> Do we have a way to reach the tens of thousands of institutions you
>>>>>> believe are there?
>>>>>>
>>>>>> My belief (just anecdotal information) is that it’s more like a few
>>>>>> hundred institutions and that out of that number, less than 10% are
>>>> on a
>>>>>> recent release.  And, of that smaller number, few, if any, are still
>>>> using
>>>>>> Collection Sheets.  I could be wrong - I’d like the data!  I’d say
>>>> that
>>>>>> vendors can show up with their data here to help make this vendor
>>>> neutral
>>>>>> space more informed.
>>>>>>
>>>>>> If group lending and group-member lending and collection sheets are
>>>> still
>>>>>> needed, then perhaps those project volunteers can contribute the
>>>> needed
>>>>>> updates to keep the functionality useful.
>>>>>>
>>>>>> Thanks
>>>>>> Thanks
>>>>>>
>>>>>>
>>>>>> On Mon, Aug 18, 2025 at 6:48 AM Sifiso Mtetwa <
>>>> sifiso@skyburgsystems.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi guys,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> This is an interesting topic. I have been wondering why, in general,
>>>> we
>>>>>>> seem to be deprecating more and more system functions. The individual
>>>>>>> collection sheet has served us well over the years and still does, If
>>>>>>> anything we could improve on its functionality by maybe adding a bulk
>>>>>>> collection sheet template with little detail compared to a full loan
>>>>>>> repayment bulk import template.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Fineract is used by tens of thousands of organisations throughout the
>>>>>>> world and most of them are not on this listing and may not have a
>>>> voice to
>>>>>>> air their concerns. Maybe we can find a way of exposing this thread
>>>> to
>>>>>>> include more voters.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *From:* James Dailey [mailto:jdailey@apache.org]
>>>>>>> *Sent:* Saturday, 16 August 2025 02:53
>>>>>>> *To:* dev@fineract.apache.org
>>>>>>> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
>>>>>>> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Ed
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I agree this needs to be discussed but it is important to acknowledge
>>>>>>> that THIS Fineract listserv is the only official (and required)
>>>> discussion
>>>>>>> space.  It is not the intention to bury anything in “an email
>>>> thread”.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> When I started up Mifos we spent a lot of time looking at Collection
>>>>>>> Sheets and designing process flows around them. I fully know that
>>>> this
>>>>>>> design direction was important back then in 2002-2006.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> However if there is no one here asking for them to be retained
>>>> besides
>>>>>>> you, then that is a sign that they have perhaps reached an end of
>>>> their
>>>>>>> utility.  Or, that the users are not actually here, which is a
>>>> different
>>>>>>> problem.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> So,… If there is a group using collection sheets in production AND
>>>> they
>>>>>>> are not on some permanent forked (old) version, then now is the time
>>>> to
>>>>>>> speak up.  Here.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Generally, I’m fairly certain we can refactor this with an eye toward
>>>>>>> extracting it from the core.  Repeating the logic in two places
>>>> makes no
>>>>>>> sense either.  Collection sheets are kind of assembled from
>>>>>>> constituent loans and savings.  The balances and due payments should
>>>> be
>>>>>>> calculated in the underlying components.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> As the system gets restructured we need to decide to keep this at
>>>> all, to
>>>>>>> keep it in a new place, or as some external concept/plug in. Why
>>>> wouldn’t
>>>>>>> we want a separate component?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Cheers
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Aug 14, 2025 at 4:16 PM Ed Cable <ed...@mifos.org> wrote:
>>>>>>>
>>>>>>> I'll leave the other thread for discussion of the API versioning and
>>>>>>> refactoring related to Collection Sheet API but wanted to create a
>>>> separate
>>>>>>> thread regarding the deprecation of the Collection Sheet.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> In general, for this and removal of any functionality, it's something
>>>>>>> that needs to be discussed openly with the community and with a vote
>>>> and
>>>>>>> not a decision buried in a mailing list thread.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> For the collection sheet specifically, more thought has to be given
>>>> to
>>>>>>> its deprecation as the centrality and highly coupled nature of the
>>>>>>> collection sheet is being understated as it isn't merely a report
>>>> that's to
>>>>>>> be printed or a form filled out via a mobile application. It's a
>>>>>>> significant portion of the user interface and highly coupled to many
>>>> of the
>>>>>>> microfinance features around groups/centers/meeting scheduling, staff
>>>>>>> assignment, etc.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I do agree that from a UI perspective, the collection sheet and other
>>>>>>> microfinance-centric functionalities and flows should be viewable
>>>> based on
>>>>>>> a configurable setting. As it doesn't lend itself to the optimal user
>>>>>>> experience for a large portion of current Fineract user base.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I also am supportive of a strategy of slimming Fineract to its core
>>>>>>> services and functionality above core Fineract services and APIs can
>>>> be
>>>>>>> extracted out.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> So I do think we should give thoughtful consideration to what
>>>> abstracting
>>>>>>> out the collection sheet and corresponding microfinance
>>>> functionality would
>>>>>>> look like and what that effort would entail to abstract it out
>>>> without
>>>>>>> adversely impacting the original user base of the software but it's
>>>> not as
>>>>>>> simple as deprecating these API.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I welcome others' thoughts and inputs as I know even with
>>>> microfinance
>>>>>>> itself, the methodology has evolved and group lending and the
>>>> concept of a
>>>>>>> collection sheet isn't as central as it once was
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Ed
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Aug 13, 2025 at 6:36 PM Kapil Panchal <
>>>>>>> kapil.panchal.development@gmail.com> wrote:
>>>>>>>
>>>>>>> Hi James,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I can proceed by marking this feature as @Deprecated and/or
>>>> performing a
>>>>>>> safe refactor to remove the API.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> For API versioning I agree to Aleksandar Vidakovic's proposal on
>>>> adapting
>>>>>>> to the SpringBoot v7/Spring Framework v4. If I may, Aleksandar
>>>>>>> Vidakovic takes the lead on this project and I can help to support
>>>> the
>>>>>>> conversion?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Kapil
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Aug 14, 2025 at 5:07 AM James Dailey <jd...@apache.org>
>>>> wrote:
>>>>>>>
>>>>>>> Hi Kapil
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I might suggest looking at this as an opportunity to remove the
>>>>>>> collection sheet entirely from the Fineract namespace.  It’s a legacy
>>>>>>> concept I and others designed a long time ago, originally in 2002
>>>> based on
>>>>>>> collection sheets we gathered from a dozen countries. It is strongly
>>>> tied
>>>>>>> to concepts in microfinance field operations, and especially when
>>>> there was
>>>>>>> no data connectivity.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> It belongs perhaps as a sort of external microservice - data loading
>>>> via
>>>>>>> a bulk import could still be enabled.
>>>>>>>
>>>>>>> The API versioning is a good idea but needs to be more holistic
>>>> across
>>>>>>> the platform I think.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <ad...@gmail.com>
>>>> wrote:
>>>>>>>
>>>>>>> Hi Kapil,
>>>>>>>
>>>>>>> Thank you for raising the concerns below. I’ll need some additional
>>>>>>> details to fully understand your points:
>>>>>>>
>>>>>>> 1.      *Collection Sheet API* – You mentioned it appears
>>>> non-functional
>>>>>>> and contains several logical errors.
>>>>>>>
>>>>>>> o    If it’s indeed not working, that’s a separate, high-priority
>>>>>>> discussion.
>>>>>>>
>>>>>>> o    Could you clarify which logical errors you were referring to,
>>>> and
>>>>>>> what specifically makes you think it’s non-functional?
>>>>>>>
>>>>>>> 2.      *Service annotations* – You noted that service methods are
>>>> not
>>>>>>> annotated with @Service and that beans are defined manually.
>>>>>>>
>>>>>>> o    Are you referring to the
>>>>>>> CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being
>>>> defined
>>>>>>> via configuration?
>>>>>>>
>>>>>>> 3.      *Repository wrappers annotated with **@Service* – You
>>>> mentioned
>>>>>>> that this mandates full unit test coverage but that they should
>>>> ideally be
>>>>>>> annotated with @Component.
>>>>>>>
>>>>>>> o    Could you point out the exact classes you had in mind?
>>>>>>>
>>>>>>> As for the other points, I agree we can refactor and remove redundant
>>>>>>> logic—please feel free to suggest specific improvements or start
>>>> work on
>>>>>>> them immediately!
>>>>>>>
>>>>>>> However, be careful by moving anything into the fineract-core… We are
>>>>>>> aiming to keep it as small as possible as everything is built on top
>>>> of
>>>>>>> this module! If collection sheet are used for loans and savings - for
>>>>>>> example - than the recommended move is NOT to move this logic into
>>>> core!
>>>>>>>
>>>>>>> Either:
>>>>>>>
>>>>>>> - we split the logic into fineract-loan and fineract-savings
>>>>>>>
>>>>>>> - Move the logic into a new module
>>>>>>>
>>>>>>> - Leave it in fineract-provider for now
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Shall you have any questions, please let us know!
>>>>>>>
>>>>>>> Regards,
>>>>>>> Adam
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 2025. Aug 11., at 12:09, Kapil Panchal <
>>>>>>> kapil.panchal.development@gmail.com> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Hi Adam,
>>>>>>>
>>>>>>> I’m currently working on *FINERACT-2290* and have a few questions
>>>> before
>>>>>>> I submit a pull request.
>>>>>>>
>>>>>>> The *Collection Sheet API* in its current state appears
>>>> non-functional
>>>>>>> and contains several logical errors. It seems there was an earlier
>>>> attempt
>>>>>>> to convert from a JSON string request parameter to a class-based
>>>> request
>>>>>>> object, but:
>>>>>>>
>>>>>>> Certain fields are missing.
>>>>>>>
>>>>>>> The serializer is not correctly populating the objects, which causes
>>>> the
>>>>>>> conditional checks to be bypassed and results in incorrect (false)
>>>>>>> responses.
>>>>>>>
>>>>>>> This change set is *high risk* because it touches most of the loan
>>>> and
>>>>>>> savings product logic. I’ve had to refactor almost all major methods.
>>>>>>> Extensive integration and end-to-end testing will be required to
>>>> ensure
>>>>>>> there are no regressions, especially in edge cases. At present,
>>>> there are
>>>>>>> no unit or integration tests for this functionality, and test
>>>> creation is
>>>>>>> outside the current ticket scope. I’ve been iterating on this for a
>>>> while,
>>>>>>> and only today have I reached a stable state after several
>>>> experimental and
>>>>>>> build-breaking attempts.
>>>>>>>
>>>>>>> *Key Observations:*
>>>>>>>
>>>>>>> Service methods are not annotated with @Service; instead, beans are
>>>>>>> defined manually.
>>>>>>>
>>>>>>> Repository wrappers are annotated with @Service. This mandates full
>>>> unit
>>>>>>> test coverage for these methods, but they should ideally be
>>>> annotated with
>>>>>>> @Component.
>>>>>>>
>>>>>>> I agree with prior discussions on separating bean validation —
>>>> having a
>>>>>>> dedicated @Component validation class allows the request object to
>>>> handle
>>>>>>> checks independent of database queries.
>>>>>>>
>>>>>>> Validation components can also perform database-related validations;
>>>>>>> these can be injected into service classes for cleaner architecture.
>>>>>>>
>>>>>>> Such validation components should be placed in *Fineract-Core* so
>>>> they
>>>>>>> are reusable across modules, reducing future refactoring needs.
>>>>>>>
>>>>>>> The current design of having commands in *Fineract-Core* and
>>>>>>> handlers/services/repositories in respective modules is good — it
>>>> cleanly
>>>>>>> decouples command definition from execution.
>>>>>>>
>>>>>>> There is extensive use of this. in singleton contexts (API, Service,
>>>>>>> Repository). While not harmful, it’s unnecessary boilerplate.
>>>>>>>
>>>>>>> Multiple redundant intermediate DTOs exist where the request DTO
>>>> itself
>>>>>>> could be reused for data transfer.
>>>>>>>
>>>>>>> I found redundant logic — e.g., a for loop with a break statement
>>>> that
>>>>>>> effectively executes only once; this can be simplified.
>>>>>>>
>>>>>>> Some JDBC template queries use reserved SQL keywords, causing
>>>> exceptions.
>>>>>>> Refactoring these queries resolves the issue and returns proper
>>>> response
>>>>>>> objects.
>>>>>>>
>>>>>>> *Suggestions:*
>>>>>>>
>>>>>>> *Where appropriate, large tickets should be broken into subtasks to
>>>>>>> manage complexity and reviewability.*
>>>>>>>
>>>>>>> It may help to have a dedicated *developer-only Slack channel *for
>>>>>>> technical discussions. This could complement other community spaces
>>>> if
>>>>>>> there’s a need to keep certain conversations more focused.
>>>>>>>
>>>>>>> What are your thoughts on the above?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Kapil
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> *Ed Cable*
>>>>>>>
>>>>>>> President/CEO, Mifos Initiative
>>>>>>>
>>>>>>> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Collectively Creating a World of 3 Billion Maries | *
>>>> http://mifos.org
>>>>>>> <http://facebook.com/mifos>  <http://www.twitter.com/mifos>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>>
> 

Re: Mission of Fineract discussion (was Re: Collection Sheet Deprecation ...)

[Paul <pc...@gmail.com>]

Great feedback, Felix, Zayyad.
I would like to share a perspective from history and why the big picture
approach and broader mission is so important.

There was a phrase in use when I was a baby. "Separate, but Equal." This
was a phrase used to justify keeping US school systems segregated by race.
"Separate but Equal" was a lie. The systems were never equal in funding,
quality or anything else.
Finance and access is the same.

*What does that have to do with Fineract?*
Mission solely dedicated to the unbanked shrinks the number of people
willing to join the community.
Others may be sympathetic, but won't directly participate by contributing.
I happen to be interested in the unbanked and personally support that
mission.

HOWEVER, my main interest in Fineract is to support a commercial enterprise
Open Source core banking system. A core banking system which integrates
embedds services anywhere, not just microfinance and money transfers.

When the Fineract mission was focused on the unbanked, it was seriously
under participated IMO. With its slight change in mission which is still
inclusive of the underbanked, participation is increasing and releases are
beginning to become more regular. The unbanked needs to be supported with a
system capable of full integration with the rest of the world, not a
"separate but equal system".

I hope my history story makes sense to at least support my POV.

*How Fineract works for the unbanked with its current mission statement.  *

MIFOS is solely dedicated to the underserved and has the frontend to do so.
In my opinion, if collection sheets are still used in working with
unbanked, then MIFOS community should adopt and help maintain them them as
a current, modern, secure function so that the MIFOS frontend works with
the Fineract backend. MIFOS like all other users of Fineract get the
benefit of a stronger core banking system and vendors with dedicated
purpose work together to maintain their particular core channels within
Fineract.

The image is a generalization to convey the concept. (It may not be precise
as to the vendors and players, but should reinforce my POV.)
So, you, your interest or company wants a function or to maintain an
existing function? GREAT!
Step up! Own that solution within Fineract.
[image: image.png]
I speak solely for myself and my POV, but this approach will generate 10X
deeper support than segregated methods.

In the past, there was insufficient community, but I think there was also
"community orchestration" missing to a degree.

*Fail Model:*
Hoping it all gets done and telling the community at large "do something"
doesn't work.

*Success Model: *
When a platform is useful across a whole ecosystem, then it can be jointly
maintained with nominal management. People are willing to help. Just like
musicians want to play music, they want to work together. In RARE
circumstances, the community is so SINGLE minded, the build, like JAZZ,
happens spontaneously and all agree, its really cool.

*However, the bulk of the world needs direction and orchestration* to "make
music" of product delivery. Because of those many different interests,
Fineract needs process to harmonize and create results.

In the end, there also must be a binary decision, "yes or no" to support an
issue. It comes down to; " will a community member support a function
important to their channel?"

I'm not a developer, my useful skills are domain knowledge and
instructional orchestration (process). So I challenge all community members
to find a channel required to fulfill your channel or interest and
volunteer to "own it" through a date certain. Better yet, form a small team
of like minds and assure that function is ALWAYS current and secure.

This is a challenge to all community members and targeting no one.

Paul

On Sat, Aug 30, 2025 at 3:34 PM James Dailey <jd...@apache.org> wrote:

> Zayyad, Felix and others -
>
> Indeed.  And all of this was laid out for many years as the strategy.
>
> When I was on the Board of Mifos (the non profit, not the project) we
> decided to contribute the MifosX code to the ASF to ensure both:
>   a) a broader financial services industry approach, and
>   b) a more established foundational home for the core of the project.
>
> There were other reasons, but those remain primary in my mind.
>
> The concept was that Mifos org, which is a Vendor in the ASF parlance,
> would continue its Financial Inclusion mission and make sure that the
> releases that were coming out of the project would retain the flavor and
> intent of the original mifos project (which I started and designed in
> 2001).  The Mifos AdminUI /CommunityApp open source front end UI gave it a
> place in the ecosystem and a way to ensure that alignment.
>
> But, importantly, when we made this decision, we deliberately wanted to
> see more financial institutions and fintechs use the backend system, AND
> vitally, to help maintain it, with contributions upstream.  That is the
> sign of a healthy open source project.
>
> I don't speak for Mifos, but I believe that the vision at Mifos remains
> about financial inclusion.   But the vision of Fineract and the mission of
> the project is different.  We actually VOTED on this revised statement as a
> community, I think five years ago.
>
> So, if you are seeing a difference between Mifos and Fineract, then we
> have succeeded in an important goal.  The two are NOT THE SAME.  This is
> NOT mifos.
>
> This has been called out in many many Reports to the ASF Board.  See the
> wiki pages for those.  The key theme is Vendor neutrality - we do not
> privilege any vendor.
>
> And, I would argue that this is the right healthy direction for this
> Fineract project.  Microfinance serves an important role in the world, and
> continues despite many expecting it to decline or go away.  And, the
> mission of the original concept of Mifos was not about Microfinance but
> about financial inclusion.  However that happens.
>
> And, fundamentally, I personally remain focused on this - disrupting the
> financial industry with open source software that can functionally be the
> basis of any bank or fintech globally.  It should lower the cost of
> financial inclusion, it should do that via any kind of institution at a
> structural level.  Unbanked and underbanked are included.
>
> So, if you are not seeing what you want here, then you can invest more
> time and resources to make it more of what you want, but I would also
> suggest that the fundamental thing is to have healthy vendors that move the
> project forward with lots of "built on top of" functionality that serve a
> wide range of financial institutions.  If those components are also open
> source, that's ok, but it isn't essential.
>
> ** Deprecation process **
> Deprecating features is a natural part of any system evolution - we need
> to prune the tree from time to time.  If users are demanding those features
> then they should speak up - as this thread illustrated - and mechanisms
> need to be put in place to "protect them".  Here of course, I am referring
> to my last email and testing, to start with.
>
> And, we should do a better job of documenting the product roadmap, and
> having clear discussions about deprecating features - Before they are
> deprecated.
>
> That starts with you listing your required feature set and documenting it
> somewhere perhaps - or relying on the Vendor to speak up for your interests
> if you don't.
>
> And, this is a good discussion to have.  Perhaps a community call on this
> topic would be welcomed in coming weeks.
>
> James
>
>
>
> On Sat, Aug 30, 2025 at 3:13 AM Zayyad A. Said <
> zayyad@intrasofttechnologies.com> wrote:
>
>> Hello Fineract community,
>>
>> I concur with the observations below from Felix.
>>
>> We have been implementing Mifos since 2011 and we have seen the project
>> transitioning from Mifos 2.x to Mifos X and now with the new developments.
>> The current focus seems to have shifted from the original mission of
>> "Creating a world of 3 Billion Maries where each of the 2 billion poor and
>> unbanked has access to the financial resources needed to create a better
>> life for themselves and their family."
>>
>> Group lending has been used for decades and Mifos was originally intended
>> for and served the model, the new features being introduced are geared
>> towards commercial use. We saw this coming and we had to maintain old fork
>> for our MFI clients where we continue to improve based on their needs.
>>
>> While I have no problem with having new features, I seem not to
>> understand why we want to deprecate features of group lending that are
>> still being used even if with the minority of users globally yet they do no
>> harm to the security and overall performance of the project.
>>
>> I suggest the leadership to rethink the idea of deprecating collection
>> sheet functionality and other key features used in group lending.
>>
>> Best Regards,
>>
>> Zayyad A. Said
>> Intrasoft Technologies Limited
>>
>>  *********
>>
>> *Zayyad A. Said | Chief Executive Officer*
>>
>> Suite H32, Delamere Flats - Milimani Road, Nairobi.
>>
>> Cell No.: +254 716 615274 | Skype: *zsaid2011*
>>
>> Email: zayyad@intrasofttechnologies.com
>>
>> Schedule Meetings: https://calendly.com/zayyadsaid
>>
>> <https://calendly.com/zayyadsaid>
>>
>>
>> *" You can achieve what you want if you just help enough others get what
>> they want.."*
>>
>> *Note: Save the environment, please don't print this email unless its
>> really necessary for you to do that.*
>>
>> On Sat, Aug 30, 2025, 11:20 Felix van Hove <fv...@gmx.de.invalid>
>> wrote:
>>
>>> Let me chip in with a more personal perspective.
>>>
>>> I'm following the development of the Mifos web app and - to a certain
>>> extend - Fineract for a couple of months only. My impression from the
>>> influx of new features in Fineract is that these are predominantly not
>>> for microfinance or small entities, which Mifos once set out to support.
>>> In fact - Fineract only states that its efforts are also "including the
>>> unbanked and underbanked" - not its focus.
>>>
>>> I see the deprecation of the Collection Sheet feature in this context.
>>> How many new features have been added in the last 12 months that help
>>> the "unbanked"? How many have been added that help larger commercial
>>> entities and are of no benefit for "the unbanked"?
>>>
>>> My personal contributions depend solely on Mifos/Fineract supporting
>>> efforts for financial inclusion. I regard it as of utter importance
>>> respective features don't come as a plugin, but first class citizens.
>>>
>>> My knowledge regarding fintech is very limited. If I'm wrong or
>>> misrespresent things, please tell me.
>>>
>>> Felix
>>>
>>> On 29/08/2025 21:01, James Dailey wrote:
>>> > Devs -  Are we done with this?
>>> >
>>> > First, I am NOT ruling out this functionality as important but if no
>>> one is
>>> > actually demanding it, and no one is willing to maintain it, then it
>>> > doesn't get maintained.  That is the nature of open source projects.
>>> >
>>> > Second, it can happen that an outside vendor (e.g. Mifos) that relies
>>> > on this Collection Sheet functionality (or any functionality)
>>> inadvertently
>>> > drops that from their front end release.  Because the tests at
>>> Fineract do
>>> > not cover this fully, no one at the Vendor and no one at the Fineract
>>> > project will see the functionality fail in a build until a customer or
>>> > implementer notices. So, tests are vital for ensuring ongoing
>>> > maintainability.
>>> >
>>> > Third, if this type of thing isn't "naturally" part of the core - then
>>> it
>>> > will make a lot more sense to have it be an outside extension - in
>>> which
>>> > case the tests have to be written for the API calls, and a Vendor
>>> should
>>> > try to get such tests contributed.   I could be wrong about this, I
>>> would
>>> > be happy to debate where this belongs.
>>> >
>>> > @Edward Cable <ed...@mifos.org> - what is the assessment and status
>>> of
>>> > needed functionality and where do you think it should live?
>>> >
>>> > Thanks,
>>> > James
>>> >
>>> > On Mon, Aug 18, 2025 at 8:21 AM James Dailey <jd...@apache.org>
>>> wrote:
>>> >
>>> >> Sifiso -
>>> >>
>>> >> Do we have a way to reach the tens of thousands of institutions you
>>> >> believe are there?
>>> >>
>>> >> My belief (just anecdotal information) is that it’s more like a few
>>> >> hundred institutions and that out of that number, less than 10% are
>>> on a
>>> >> recent release.  And, of that smaller number, few, if any, are still
>>> using
>>> >> Collection Sheets.  I could be wrong - I’d like the data!  I’d say
>>> that
>>> >> vendors can show up with their data here to help make this vendor
>>> neutral
>>> >> space more informed.
>>> >>
>>> >> If group lending and group-member lending and collection sheets are
>>> still
>>> >> needed, then perhaps those project volunteers can contribute the
>>> needed
>>> >> updates to keep the functionality useful.
>>> >>
>>> >> Thanks
>>> >> Thanks
>>> >>
>>> >>
>>> >> On Mon, Aug 18, 2025 at 6:48 AM Sifiso Mtetwa <
>>> sifiso@skyburgsystems.org>
>>> >> wrote:
>>> >>
>>> >>> Hi guys,
>>> >>>
>>> >>>
>>> >>>
>>> >>> This is an interesting topic. I have been wondering why, in general,
>>> we
>>> >>> seem to be deprecating more and more system functions. The individual
>>> >>> collection sheet has served us well over the years and still does, If
>>> >>> anything we could improve on its functionality by maybe adding a bulk
>>> >>> collection sheet template with little detail compared to a full loan
>>> >>> repayment bulk import template.
>>> >>>
>>> >>>
>>> >>>
>>> >>> Fineract is used by tens of thousands of organisations throughout the
>>> >>> world and most of them are not on this listing and may not have a
>>> voice to
>>> >>> air their concerns. Maybe we can find a way of exposing this thread
>>> to
>>> >>> include more voters.
>>> >>>
>>> >>>
>>> >>>
>>> >>> Regards,
>>> >>>
>>> >>>
>>> >>>
>>> >>> *From:* James Dailey [mailto:jdailey@apache.org]
>>> >>> *Sent:* Saturday, 16 August 2025 02:53
>>> >>> *To:* dev@fineract.apache.org
>>> >>> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
>>> >>> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
>>> >>>
>>> >>>
>>> >>>
>>> >>> Ed
>>> >>>
>>> >>>
>>> >>>
>>> >>> I agree this needs to be discussed but it is important to acknowledge
>>> >>> that THIS Fineract listserv is the only official (and required)
>>> discussion
>>> >>> space.  It is not the intention to bury anything in “an email
>>> thread”.
>>> >>>
>>> >>>
>>> >>>
>>> >>> When I started up Mifos we spent a lot of time looking at Collection
>>> >>> Sheets and designing process flows around them. I fully know that
>>> this
>>> >>> design direction was important back then in 2002-2006.
>>> >>>
>>> >>>
>>> >>>
>>> >>> However if there is no one here asking for them to be retained
>>> besides
>>> >>> you, then that is a sign that they have perhaps reached an end of
>>> their
>>> >>> utility.  Or, that the users are not actually here, which is a
>>> different
>>> >>> problem.
>>> >>>
>>> >>>
>>> >>>
>>> >>> So,… If there is a group using collection sheets in production AND
>>> they
>>> >>> are not on some permanent forked (old) version, then now is the time
>>> to
>>> >>> speak up.  Here.
>>> >>>
>>> >>>
>>> >>>
>>> >>> Generally, I’m fairly certain we can refactor this with an eye toward
>>> >>> extracting it from the core.  Repeating the logic in two places
>>> makes no
>>> >>> sense either.  Collection sheets are kind of assembled from
>>> >>> constituent loans and savings.  The balances and due payments should
>>> be
>>> >>> calculated in the underlying components.
>>> >>>
>>> >>>
>>> >>>
>>> >>> As the system gets restructured we need to decide to keep this at
>>> all, to
>>> >>> keep it in a new place, or as some external concept/plug in. Why
>>> wouldn’t
>>> >>> we want a separate component?
>>> >>>
>>> >>>
>>> >>>
>>> >>> Cheers
>>> >>>
>>> >>>
>>> >>>
>>> >>> On Thu, Aug 14, 2025 at 4:16 PM Ed Cable <ed...@mifos.org> wrote:
>>> >>>
>>> >>> I'll leave the other thread for discussion of the API versioning and
>>> >>> refactoring related to Collection Sheet API but wanted to create a
>>> separate
>>> >>> thread regarding the deprecation of the Collection Sheet.
>>> >>>
>>> >>>
>>> >>>
>>> >>> In general, for this and removal of any functionality, it's something
>>> >>> that needs to be discussed openly with the community and with a vote
>>> and
>>> >>> not a decision buried in a mailing list thread.
>>> >>>
>>> >>>
>>> >>>
>>> >>> For the collection sheet specifically, more thought has to be given
>>> to
>>> >>> its deprecation as the centrality and highly coupled nature of the
>>> >>> collection sheet is being understated as it isn't merely a report
>>> that's to
>>> >>> be printed or a form filled out via a mobile application. It's a
>>> >>> significant portion of the user interface and highly coupled to many
>>> of the
>>> >>> microfinance features around groups/centers/meeting scheduling, staff
>>> >>> assignment, etc.
>>> >>>
>>> >>>
>>> >>>
>>> >>> I do agree that from a UI perspective, the collection sheet and other
>>> >>> microfinance-centric functionalities and flows should be viewable
>>> based on
>>> >>> a configurable setting. As it doesn't lend itself to the optimal user
>>> >>> experience for a large portion of current Fineract user base.
>>> >>>
>>> >>>
>>> >>>
>>> >>> I also am supportive of a strategy of slimming Fineract to its core
>>> >>> services and functionality above core Fineract services and APIs can
>>> be
>>> >>> extracted out.
>>> >>>
>>> >>>
>>> >>>
>>> >>> So I do think we should give thoughtful consideration to what
>>> abstracting
>>> >>> out the collection sheet and corresponding microfinance
>>> functionality would
>>> >>> look like and what that effort would entail to abstract it out
>>> without
>>> >>> adversely impacting the original user base of the software but it's
>>> not as
>>> >>> simple as deprecating these API.
>>> >>>
>>> >>>
>>> >>>
>>> >>> I welcome others' thoughts and inputs as I know even with
>>> microfinance
>>> >>> itself, the methodology has evolved and group lending and the
>>> concept of a
>>> >>> collection sheet isn't as central as it once was
>>> >>>
>>> >>>
>>> >>>
>>> >>> Thanks,
>>> >>>
>>> >>>
>>> >>>
>>> >>> Ed
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> On Wed, Aug 13, 2025 at 6:36 PM Kapil Panchal <
>>> >>> kapil.panchal.development@gmail.com> wrote:
>>> >>>
>>> >>> Hi James,
>>> >>>
>>> >>>
>>> >>>
>>> >>> I can proceed by marking this feature as @Deprecated and/or
>>> performing a
>>> >>> safe refactor to remove the API.
>>> >>>
>>> >>>
>>> >>>
>>> >>> For API versioning I agree to Aleksandar Vidakovic's proposal on
>>> adapting
>>> >>> to the SpringBoot v7/Spring Framework v4. If I may, Aleksandar
>>> >>> Vidakovic takes the lead on this project and I can help to support
>>> the
>>> >>> conversion?
>>> >>>
>>> >>>
>>> >>>
>>> >>> Thanks,
>>> >>>
>>> >>> Kapil
>>> >>>
>>> >>>
>>> >>>
>>> >>> On Thu, Aug 14, 2025 at 5:07 AM James Dailey <jd...@apache.org>
>>> wrote:
>>> >>>
>>> >>> Hi Kapil
>>> >>>
>>> >>>
>>> >>>
>>> >>> I might suggest looking at this as an opportunity to remove the
>>> >>> collection sheet entirely from the Fineract namespace.  It’s a legacy
>>> >>> concept I and others designed a long time ago, originally in 2002
>>> based on
>>> >>> collection sheets we gathered from a dozen countries. It is strongly
>>> tied
>>> >>> to concepts in microfinance field operations, and especially when
>>> there was
>>> >>> no data connectivity.
>>> >>>
>>> >>>
>>> >>>
>>> >>> It belongs perhaps as a sort of external microservice - data loading
>>> via
>>> >>> a bulk import could still be enabled.
>>> >>>
>>> >>> The API versioning is a good idea but needs to be more holistic
>>> across
>>> >>> the platform I think.
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <ad...@gmail.com>
>>> wrote:
>>> >>>
>>> >>> Hi Kapil,
>>> >>>
>>> >>> Thank you for raising the concerns below. I’ll need some additional
>>> >>> details to fully understand your points:
>>> >>>
>>> >>> 1.      *Collection Sheet API* – You mentioned it appears
>>> non-functional
>>> >>> and contains several logical errors.
>>> >>>
>>> >>> o    If it’s indeed not working, that’s a separate, high-priority
>>> >>> discussion.
>>> >>>
>>> >>> o    Could you clarify which logical errors you were referring to,
>>> and
>>> >>> what specifically makes you think it’s non-functional?
>>> >>>
>>> >>> 2.      *Service annotations* – You noted that service methods are
>>> not
>>> >>> annotated with @Service and that beans are defined manually.
>>> >>>
>>> >>> o    Are you referring to the
>>> >>> CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being
>>> defined
>>> >>> via configuration?
>>> >>>
>>> >>> 3.      *Repository wrappers annotated with **@Service* – You
>>> mentioned
>>> >>> that this mandates full unit test coverage but that they should
>>> ideally be
>>> >>> annotated with @Component.
>>> >>>
>>> >>> o    Could you point out the exact classes you had in mind?
>>> >>>
>>> >>> As for the other points, I agree we can refactor and remove redundant
>>> >>> logic—please feel free to suggest specific improvements or start
>>> work on
>>> >>> them immediately!
>>> >>>
>>> >>> However, be careful by moving anything into the fineract-core… We are
>>> >>> aiming to keep it as small as possible as everything is built on top
>>> of
>>> >>> this module! If collection sheet are used for loans and savings - for
>>> >>> example - than the recommended move is NOT to move this logic into
>>> core!
>>> >>>
>>> >>> Either:
>>> >>>
>>> >>> - we split the logic into fineract-loan and fineract-savings
>>> >>>
>>> >>> - Move the logic into a new module
>>> >>>
>>> >>> - Leave it in fineract-provider for now
>>> >>>
>>> >>>
>>> >>>
>>> >>> Shall you have any questions, please let us know!
>>> >>>
>>> >>> Regards,
>>> >>> Adam
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> On 2025. Aug 11., at 12:09, Kapil Panchal <
>>> >>> kapil.panchal.development@gmail.com> wrote:
>>> >>>
>>> >>>
>>> >>>
>>> >>> Hi Adam,
>>> >>>
>>> >>> I’m currently working on *FINERACT-2290* and have a few questions
>>> before
>>> >>> I submit a pull request.
>>> >>>
>>> >>> The *Collection Sheet API* in its current state appears
>>> non-functional
>>> >>> and contains several logical errors. It seems there was an earlier
>>> attempt
>>> >>> to convert from a JSON string request parameter to a class-based
>>> request
>>> >>> object, but:
>>> >>>
>>> >>> Certain fields are missing.
>>> >>>
>>> >>> The serializer is not correctly populating the objects, which causes
>>> the
>>> >>> conditional checks to be bypassed and results in incorrect (false)
>>> >>> responses.
>>> >>>
>>> >>> This change set is *high risk* because it touches most of the loan
>>> and
>>> >>> savings product logic. I’ve had to refactor almost all major methods.
>>> >>> Extensive integration and end-to-end testing will be required to
>>> ensure
>>> >>> there are no regressions, especially in edge cases. At present,
>>> there are
>>> >>> no unit or integration tests for this functionality, and test
>>> creation is
>>> >>> outside the current ticket scope. I’ve been iterating on this for a
>>> while,
>>> >>> and only today have I reached a stable state after several
>>> experimental and
>>> >>> build-breaking attempts.
>>> >>>
>>> >>> *Key Observations:*
>>> >>>
>>> >>> Service methods are not annotated with @Service; instead, beans are
>>> >>> defined manually.
>>> >>>
>>> >>> Repository wrappers are annotated with @Service. This mandates full
>>> unit
>>> >>> test coverage for these methods, but they should ideally be
>>> annotated with
>>> >>> @Component.
>>> >>>
>>> >>> I agree with prior discussions on separating bean validation —
>>> having a
>>> >>> dedicated @Component validation class allows the request object to
>>> handle
>>> >>> checks independent of database queries.
>>> >>>
>>> >>> Validation components can also perform database-related validations;
>>> >>> these can be injected into service classes for cleaner architecture.
>>> >>>
>>> >>> Such validation components should be placed in *Fineract-Core* so
>>> they
>>> >>> are reusable across modules, reducing future refactoring needs.
>>> >>>
>>> >>> The current design of having commands in *Fineract-Core* and
>>> >>> handlers/services/repositories in respective modules is good — it
>>> cleanly
>>> >>> decouples command definition from execution.
>>> >>>
>>> >>> There is extensive use of this. in singleton contexts (API, Service,
>>> >>> Repository). While not harmful, it’s unnecessary boilerplate.
>>> >>>
>>> >>> Multiple redundant intermediate DTOs exist where the request DTO
>>> itself
>>> >>> could be reused for data transfer.
>>> >>>
>>> >>> I found redundant logic — e.g., a for loop with a break statement
>>> that
>>> >>> effectively executes only once; this can be simplified.
>>> >>>
>>> >>> Some JDBC template queries use reserved SQL keywords, causing
>>> exceptions.
>>> >>> Refactoring these queries resolves the issue and returns proper
>>> response
>>> >>> objects.
>>> >>>
>>> >>> *Suggestions:*
>>> >>>
>>> >>> *Where appropriate, large tickets should be broken into subtasks to
>>> >>> manage complexity and reviewability.*
>>> >>>
>>> >>> It may help to have a dedicated *developer-only Slack channel *for
>>> >>> technical discussions. This could complement other community spaces
>>> if
>>> >>> there’s a need to keep certain conversations more focused.
>>> >>>
>>> >>> What are your thoughts on the above?
>>> >>>
>>> >>> Thanks,
>>> >>> Kapil
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>>
>>> >>> *Ed Cable*
>>> >>>
>>> >>> President/CEO, Mifos Initiative
>>> >>>
>>> >>> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>>> >>>
>>> >>>
>>> >>>
>>> >>> *Collectively Creating a World of 3 Billion Maries | *
>>> http://mifos.org
>>> >>> <http://facebook.com/mifos>  <http://www.twitter.com/mifos>
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >
>>>
>>>

-- 
--
Paul

Re: Mission of Fineract discussion (was Re: Collection Sheet Deprecation ...)

[Felix van Hove <fv...@gmx.de.INVALID>]

Thank you, James, for this clarification!

 >  The key theme is Vendor neutrality - we do not privilege any vendor.

You yourself convinced me a while ago successfully that Mifos represents 
a vendor in the Apache framework, which aims at a maximum of 
independence of its projects. Not easy for me to swallow - if I judge 
Fineract developments according to Mifos standards, in some sense it's 
infringing Apache's autonomy - and I say this without irony. But I feel 
like exclaiming "The same applies to the company you are heading and any 
other company tailoring Fineract to client needs!".

I can really only speak from the perspective of the web app project 
here, and might infer things too quickly. But since I work on the Mifos 
app, all the larger features contributed by companies seem to come from 
client projects (often a project of a single client); and without 
alignment to some pre-existing vendor independent Apache planning they 
translate to Fineract and subsequently Mifos web app features. Were 
these features set out in some Fineract roadmap and then developed 
accordingly? Or were they coming from client requirements and from there 
found their way in both of our repositories? Just two examples:

https://issues.apache.org/jira/browse/FINERACT-2311
https://issues.apache.org/jira/browse/FINERACT-2232

Please correct me, if these examples are wrong or I misrepresent what is 
happening. In any case, these are probably valuable contributions. But 
it shouldn't be possible that commercial interests drive development 
unhitched to any ideology, and at the same time people who are in favor 
of clear-cut means of financial inclusion are turned down as not being 
vendor neutral.

I suggest to have a roadmap discussion *before* features like Collection 
Sheets are deprecated. And make sure that *everyone* has a say in this 
discussion, including those who seek to include the unbanked.

Felix



On 30/08/2025 22:33, James Dailey wrote:
> Zayyad, Felix and others -
> 
> Indeed.  And all of this was laid out for many years as the strategy.
> 
> When I was on the Board of Mifos (the non profit, not the project) we
> decided to contribute the MifosX code to the ASF to ensure both:
>    a) a broader financial services industry approach, and
>    b) a more established foundational home for the core of the project.
> 
> There were other reasons, but those remain primary in my mind.
> 
> The concept was that Mifos org, which is a Vendor in the ASF parlance,
> would continue its Financial Inclusion mission and make sure that the
> releases that were coming out of the project would retain the flavor and
> intent of the original mifos project (which I started and designed in
> 2001).  The Mifos AdminUI /CommunityApp open source front end UI gave it a
> place in the ecosystem and a way to ensure that alignment.
> 
> But, importantly, when we made this decision, we deliberately wanted to see
> more financial institutions and fintechs use the backend system, AND
> vitally, to help maintain it, with contributions upstream.  That is the
> sign of a healthy open source project.
> 
> I don't speak for Mifos, but I believe that the vision at Mifos remains
> about financial inclusion.   But the vision of Fineract and the mission of
> the project is different.  We actually VOTED on this revised statement as a
> community, I think five years ago.
> 
> So, if you are seeing a difference between Mifos and Fineract, then we have
> succeeded in an important goal.  The two are NOT THE SAME.  This is NOT
> mifos.
> 
> This has been called out in many many Reports to the ASF Board.  See the
> wiki pages for those.  The key theme is Vendor neutrality - we do not
> privilege any vendor.
> 
> And, I would argue that this is the right healthy direction for this
> Fineract project.  Microfinance serves an important role in the world, and
> continues despite many expecting it to decline or go away.  And, the
> mission of the original concept of Mifos was not about Microfinance but
> about financial inclusion.  However that happens.
> 
> And, fundamentally, I personally remain focused on this - disrupting the
> financial industry with open source software that can functionally be the
> basis of any bank or fintech globally.  It should lower the cost of
> financial inclusion, it should do that via any kind of institution at a
> structural level.  Unbanked and underbanked are included.
> 
> So, if you are not seeing what you want here, then you can invest more time
> and resources to make it more of what you want, but I would also suggest
> that the fundamental thing is to have healthy vendors that move the project
> forward with lots of "built on top of" functionality that serve a wide
> range of financial institutions.  If those components are also open source,
> that's ok, but it isn't essential.
> 
> ** Deprecation process **
> Deprecating features is a natural part of any system evolution - we need to
> prune the tree from time to time.  If users are demanding those features
> then they should speak up - as this thread illustrated - and mechanisms
> need to be put in place to "protect them".  Here of course, I am referring
> to my last email and testing, to start with.
> 
> And, we should do a better job of documenting the product roadmap, and
> having clear discussions about deprecating features - Before they are
> deprecated.
> 
> That starts with you listing your required feature set and documenting it
> somewhere perhaps - or relying on the Vendor to speak up for your interests
> if you don't.
> 
> And, this is a good discussion to have.  Perhaps a community call on this
> topic would be welcomed in coming weeks.
> 
> James
> 
> 
> 
> On Sat, Aug 30, 2025 at 3:13 AM Zayyad A. Said <
> zayyad@intrasofttechnologies.com> wrote:
> 
>> Hello Fineract community,
>>
>> I concur with the observations below from Felix.
>>
>> We have been implementing Mifos since 2011 and we have seen the project
>> transitioning from Mifos 2.x to Mifos X and now with the new developments.
>> The current focus seems to have shifted from the original mission of
>> "Creating a world of 3 Billion Maries where each of the 2 billion poor and
>> unbanked has access to the financial resources needed to create a better
>> life for themselves and their family."
>>
>> Group lending has been used for decades and Mifos was originally intended
>> for and served the model, the new features being introduced are geared
>> towards commercial use. We saw this coming and we had to maintain old fork
>> for our MFI clients where we continue to improve based on their needs.
>>
>> While I have no problem with having new features, I seem not to understand
>> why we want to deprecate features of group lending that are still
>> being used even if with the minority of users globally yet they do no harm
>> to the security and overall performance of the project.
>>
>> I suggest the leadership to rethink the idea of deprecating collection
>> sheet functionality and other key features used in group lending.
>>
>> Best Regards,
>>
>> Zayyad A. Said
>> Intrasoft Technologies Limited
>>
>>   *********
>>
>> *Zayyad A. Said | Chief Executive Officer*
>>
>> Suite H32, Delamere Flats - Milimani Road, Nairobi.
>>
>> Cell No.: +254 716 615274 | Skype: *zsaid2011*
>>
>> Email: zayyad@intrasofttechnologies.com
>>
>> Schedule Meetings: https://calendly.com/zayyadsaid
>>
>> <https://calendly.com/zayyadsaid>
>>
>>
>> *" You can achieve what you want if you just help enough others get what
>> they want.."*
>>
>> *Note: Save the environment, please don't print this email unless its
>> really necessary for you to do that.*
>>
>> On Sat, Aug 30, 2025, 11:20 Felix van Hove <fv...@gmx.de.invalid>
>> wrote:
>>
>>> Let me chip in with a more personal perspective.
>>>
>>> I'm following the development of the Mifos web app and - to a certain
>>> extend - Fineract for a couple of months only. My impression from the
>>> influx of new features in Fineract is that these are predominantly not
>>> for microfinance or small entities, which Mifos once set out to support.
>>> In fact - Fineract only states that its efforts are also "including the
>>> unbanked and underbanked" - not its focus.
>>>
>>> I see the deprecation of the Collection Sheet feature in this context.
>>> How many new features have been added in the last 12 months that help
>>> the "unbanked"? How many have been added that help larger commercial
>>> entities and are of no benefit for "the unbanked"?
>>>
>>> My personal contributions depend solely on Mifos/Fineract supporting
>>> efforts for financial inclusion. I regard it as of utter importance
>>> respective features don't come as a plugin, but first class citizens.
>>>
>>> My knowledge regarding fintech is very limited. If I'm wrong or
>>> misrespresent things, please tell me.
>>>
>>> Felix
>>>
>>> On 29/08/2025 21:01, James Dailey wrote:
>>>> Devs -  Are we done with this?
>>>>
>>>> First, I am NOT ruling out this functionality as important but if no
>>> one is
>>>> actually demanding it, and no one is willing to maintain it, then it
>>>> doesn't get maintained.  That is the nature of open source projects.
>>>>
>>>> Second, it can happen that an outside vendor (e.g. Mifos) that relies
>>>> on this Collection Sheet functionality (or any functionality)
>>> inadvertently
>>>> drops that from their front end release.  Because the tests at Fineract
>>> do
>>>> not cover this fully, no one at the Vendor and no one at the Fineract
>>>> project will see the functionality fail in a build until a customer or
>>>> implementer notices. So, tests are vital for ensuring ongoing
>>>> maintainability.
>>>>
>>>> Third, if this type of thing isn't "naturally" part of the core - then
>>> it
>>>> will make a lot more sense to have it be an outside extension - in which
>>>> case the tests have to be written for the API calls, and a Vendor should
>>>> try to get such tests contributed.   I could be wrong about this, I
>>> would
>>>> be happy to debate where this belongs.
>>>>
>>>> @Edward Cable <ed...@mifos.org> - what is the assessment and status
>>> of
>>>> needed functionality and where do you think it should live?
>>>>
>>>> Thanks,
>>>> James
>>>>
>>>> On Mon, Aug 18, 2025 at 8:21 AM James Dailey <jd...@apache.org>
>>> wrote:
>>>>
>>>>> Sifiso -
>>>>>
>>>>> Do we have a way to reach the tens of thousands of institutions you
>>>>> believe are there?
>>>>>
>>>>> My belief (just anecdotal information) is that it’s more like a few
>>>>> hundred institutions and that out of that number, less than 10% are on
>>> a
>>>>> recent release.  And, of that smaller number, few, if any, are still
>>> using
>>>>> Collection Sheets.  I could be wrong - I’d like the data!  I’d say that
>>>>> vendors can show up with their data here to help make this vendor
>>> neutral
>>>>> space more informed.
>>>>>
>>>>> If group lending and group-member lending and collection sheets are
>>> still
>>>>> needed, then perhaps those project volunteers can contribute the needed
>>>>> updates to keep the functionality useful.
>>>>>
>>>>> Thanks
>>>>> Thanks
>>>>>
>>>>>
>>>>> On Mon, Aug 18, 2025 at 6:48 AM Sifiso Mtetwa <
>>> sifiso@skyburgsystems.org>
>>>>> wrote:
>>>>>
>>>>>> Hi guys,
>>>>>>
>>>>>>
>>>>>>
>>>>>> This is an interesting topic. I have been wondering why, in general,
>>> we
>>>>>> seem to be deprecating more and more system functions. The individual
>>>>>> collection sheet has served us well over the years and still does, If
>>>>>> anything we could improve on its functionality by maybe adding a bulk
>>>>>> collection sheet template with little detail compared to a full loan
>>>>>> repayment bulk import template.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Fineract is used by tens of thousands of organisations throughout the
>>>>>> world and most of them are not on this listing and may not have a
>>> voice to
>>>>>> air their concerns. Maybe we can find a way of exposing this thread to
>>>>>> include more voters.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>>
>>>>>>
>>>>>> *From:* James Dailey [mailto:jdailey@apache.org]
>>>>>> *Sent:* Saturday, 16 August 2025 02:53
>>>>>> *To:* dev@fineract.apache.org
>>>>>> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
>>>>>> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
>>>>>>
>>>>>>
>>>>>>
>>>>>> Ed
>>>>>>
>>>>>>
>>>>>>
>>>>>> I agree this needs to be discussed but it is important to acknowledge
>>>>>> that THIS Fineract listserv is the only official (and required)
>>> discussion
>>>>>> space.  It is not the intention to bury anything in “an email thread”.
>>>>>>
>>>>>>
>>>>>>
>>>>>> When I started up Mifos we spent a lot of time looking at Collection
>>>>>> Sheets and designing process flows around them. I fully know that this
>>>>>> design direction was important back then in 2002-2006.
>>>>>>
>>>>>>
>>>>>>
>>>>>> However if there is no one here asking for them to be retained besides
>>>>>> you, then that is a sign that they have perhaps reached an end of
>>> their
>>>>>> utility.  Or, that the users are not actually here, which is a
>>> different
>>>>>> problem.
>>>>>>
>>>>>>
>>>>>>
>>>>>> So,… If there is a group using collection sheets in production AND
>>> they
>>>>>> are not on some permanent forked (old) version, then now is the time
>>> to
>>>>>> speak up.  Here.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Generally, I’m fairly certain we can refactor this with an eye toward
>>>>>> extracting it from the core.  Repeating the logic in two places makes
>>> no
>>>>>> sense either.  Collection sheets are kind of assembled from
>>>>>> constituent loans and savings.  The balances and due payments should
>>> be
>>>>>> calculated in the underlying components.
>>>>>>
>>>>>>
>>>>>>
>>>>>> As the system gets restructured we need to decide to keep this at
>>> all, to
>>>>>> keep it in a new place, or as some external concept/plug in. Why
>>> wouldn’t
>>>>>> we want a separate component?
>>>>>>
>>>>>>
>>>>>>
>>>>>> Cheers
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Aug 14, 2025 at 4:16 PM Ed Cable <ed...@mifos.org> wrote:
>>>>>>
>>>>>> I'll leave the other thread for discussion of the API versioning and
>>>>>> refactoring related to Collection Sheet API but wanted to create a
>>> separate
>>>>>> thread regarding the deprecation of the Collection Sheet.
>>>>>>
>>>>>>
>>>>>>
>>>>>> In general, for this and removal of any functionality, it's something
>>>>>> that needs to be discussed openly with the community and with a vote
>>> and
>>>>>> not a decision buried in a mailing list thread.
>>>>>>
>>>>>>
>>>>>>
>>>>>> For the collection sheet specifically, more thought has to be given to
>>>>>> its deprecation as the centrality and highly coupled nature of the
>>>>>> collection sheet is being understated as it isn't merely a report
>>> that's to
>>>>>> be printed or a form filled out via a mobile application. It's a
>>>>>> significant portion of the user interface and highly coupled to many
>>> of the
>>>>>> microfinance features around groups/centers/meeting scheduling, staff
>>>>>> assignment, etc.
>>>>>>
>>>>>>
>>>>>>
>>>>>> I do agree that from a UI perspective, the collection sheet and other
>>>>>> microfinance-centric functionalities and flows should be viewable
>>> based on
>>>>>> a configurable setting. As it doesn't lend itself to the optimal user
>>>>>> experience for a large portion of current Fineract user base.
>>>>>>
>>>>>>
>>>>>>
>>>>>> I also am supportive of a strategy of slimming Fineract to its core
>>>>>> services and functionality above core Fineract services and APIs can
>>> be
>>>>>> extracted out.
>>>>>>
>>>>>>
>>>>>>
>>>>>> So I do think we should give thoughtful consideration to what
>>> abstracting
>>>>>> out the collection sheet and corresponding microfinance functionality
>>> would
>>>>>> look like and what that effort would entail to abstract it out without
>>>>>> adversely impacting the original user base of the software but it's
>>> not as
>>>>>> simple as deprecating these API.
>>>>>>
>>>>>>
>>>>>>
>>>>>> I welcome others' thoughts and inputs as I know even with microfinance
>>>>>> itself, the methodology has evolved and group lending and the concept
>>> of a
>>>>>> collection sheet isn't as central as it once was
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>>
>>>>>>
>>>>>> Ed
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Aug 13, 2025 at 6:36 PM Kapil Panchal <
>>>>>> kapil.panchal.development@gmail.com> wrote:
>>>>>>
>>>>>> Hi James,
>>>>>>
>>>>>>
>>>>>>
>>>>>> I can proceed by marking this feature as @Deprecated and/or
>>> performing a
>>>>>> safe refactor to remove the API.
>>>>>>
>>>>>>
>>>>>>
>>>>>> For API versioning I agree to Aleksandar Vidakovic's proposal on
>>> adapting
>>>>>> to the SpringBoot v7/Spring Framework v4. If I may, Aleksandar
>>>>>> Vidakovic takes the lead on this project and I can help to support the
>>>>>> conversion?
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Kapil
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Aug 14, 2025 at 5:07 AM James Dailey <jd...@apache.org>
>>> wrote:
>>>>>>
>>>>>> Hi Kapil
>>>>>>
>>>>>>
>>>>>>
>>>>>> I might suggest looking at this as an opportunity to remove the
>>>>>> collection sheet entirely from the Fineract namespace.  It’s a legacy
>>>>>> concept I and others designed a long time ago, originally in 2002
>>> based on
>>>>>> collection sheets we gathered from a dozen countries. It is strongly
>>> tied
>>>>>> to concepts in microfinance field operations, and especially when
>>> there was
>>>>>> no data connectivity.
>>>>>>
>>>>>>
>>>>>>
>>>>>> It belongs perhaps as a sort of external microservice - data loading
>>> via
>>>>>> a bulk import could still be enabled.
>>>>>>
>>>>>> The API versioning is a good idea but needs to be more holistic across
>>>>>> the platform I think.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <ad...@gmail.com>
>>> wrote:
>>>>>>
>>>>>> Hi Kapil,
>>>>>>
>>>>>> Thank you for raising the concerns below. I’ll need some additional
>>>>>> details to fully understand your points:
>>>>>>
>>>>>> 1.      *Collection Sheet API* – You mentioned it appears
>>> non-functional
>>>>>> and contains several logical errors.
>>>>>>
>>>>>> o    If it’s indeed not working, that’s a separate, high-priority
>>>>>> discussion.
>>>>>>
>>>>>> o    Could you clarify which logical errors you were referring to, and
>>>>>> what specifically makes you think it’s non-functional?
>>>>>>
>>>>>> 2.      *Service annotations* – You noted that service methods are not
>>>>>> annotated with @Service and that beans are defined manually.
>>>>>>
>>>>>> o    Are you referring to the
>>>>>> CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being
>>> defined
>>>>>> via configuration?
>>>>>>
>>>>>> 3.      *Repository wrappers annotated with **@Service* – You
>>> mentioned
>>>>>> that this mandates full unit test coverage but that they should
>>> ideally be
>>>>>> annotated with @Component.
>>>>>>
>>>>>> o    Could you point out the exact classes you had in mind?
>>>>>>
>>>>>> As for the other points, I agree we can refactor and remove redundant
>>>>>> logic—please feel free to suggest specific improvements or start work
>>> on
>>>>>> them immediately!
>>>>>>
>>>>>> However, be careful by moving anything into the fineract-core… We are
>>>>>> aiming to keep it as small as possible as everything is built on top
>>> of
>>>>>> this module! If collection sheet are used for loans and savings - for
>>>>>> example - than the recommended move is NOT to move this logic into
>>> core!
>>>>>>
>>>>>> Either:
>>>>>>
>>>>>> - we split the logic into fineract-loan and fineract-savings
>>>>>>
>>>>>> - Move the logic into a new module
>>>>>>
>>>>>> - Leave it in fineract-provider for now
>>>>>>
>>>>>>
>>>>>>
>>>>>> Shall you have any questions, please let us know!
>>>>>>
>>>>>> Regards,
>>>>>> Adam
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 2025. Aug 11., at 12:09, Kapil Panchal <
>>>>>> kapil.panchal.development@gmail.com> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> Hi Adam,
>>>>>>
>>>>>> I’m currently working on *FINERACT-2290* and have a few questions
>>> before
>>>>>> I submit a pull request.
>>>>>>
>>>>>> The *Collection Sheet API* in its current state appears non-functional
>>>>>> and contains several logical errors. It seems there was an earlier
>>> attempt
>>>>>> to convert from a JSON string request parameter to a class-based
>>> request
>>>>>> object, but:
>>>>>>
>>>>>> Certain fields are missing.
>>>>>>
>>>>>> The serializer is not correctly populating the objects, which causes
>>> the
>>>>>> conditional checks to be bypassed and results in incorrect (false)
>>>>>> responses.
>>>>>>
>>>>>> This change set is *high risk* because it touches most of the loan and
>>>>>> savings product logic. I’ve had to refactor almost all major methods.
>>>>>> Extensive integration and end-to-end testing will be required to
>>> ensure
>>>>>> there are no regressions, especially in edge cases. At present, there
>>> are
>>>>>> no unit or integration tests for this functionality, and test
>>> creation is
>>>>>> outside the current ticket scope. I’ve been iterating on this for a
>>> while,
>>>>>> and only today have I reached a stable state after several
>>> experimental and
>>>>>> build-breaking attempts.
>>>>>>
>>>>>> *Key Observations:*
>>>>>>
>>>>>> Service methods are not annotated with @Service; instead, beans are
>>>>>> defined manually.
>>>>>>
>>>>>> Repository wrappers are annotated with @Service. This mandates full
>>> unit
>>>>>> test coverage for these methods, but they should ideally be annotated
>>> with
>>>>>> @Component.
>>>>>>
>>>>>> I agree with prior discussions on separating bean validation — having
>>> a
>>>>>> dedicated @Component validation class allows the request object to
>>> handle
>>>>>> checks independent of database queries.
>>>>>>
>>>>>> Validation components can also perform database-related validations;
>>>>>> these can be injected into service classes for cleaner architecture.
>>>>>>
>>>>>> Such validation components should be placed in *Fineract-Core* so they
>>>>>> are reusable across modules, reducing future refactoring needs.
>>>>>>
>>>>>> The current design of having commands in *Fineract-Core* and
>>>>>> handlers/services/repositories in respective modules is good — it
>>> cleanly
>>>>>> decouples command definition from execution.
>>>>>>
>>>>>> There is extensive use of this. in singleton contexts (API, Service,
>>>>>> Repository). While not harmful, it’s unnecessary boilerplate.
>>>>>>
>>>>>> Multiple redundant intermediate DTOs exist where the request DTO
>>> itself
>>>>>> could be reused for data transfer.
>>>>>>
>>>>>> I found redundant logic — e.g., a for loop with a break statement that
>>>>>> effectively executes only once; this can be simplified.
>>>>>>
>>>>>> Some JDBC template queries use reserved SQL keywords, causing
>>> exceptions.
>>>>>> Refactoring these queries resolves the issue and returns proper
>>> response
>>>>>> objects.
>>>>>>
>>>>>> *Suggestions:*
>>>>>>
>>>>>> *Where appropriate, large tickets should be broken into subtasks to
>>>>>> manage complexity and reviewability.*
>>>>>>
>>>>>> It may help to have a dedicated *developer-only Slack channel *for
>>>>>> technical discussions. This could complement other community spaces if
>>>>>> there’s a need to keep certain conversations more focused.
>>>>>>
>>>>>> What are your thoughts on the above?
>>>>>>
>>>>>> Thanks,
>>>>>> Kapil
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> *Ed Cable*
>>>>>>
>>>>>> President/CEO, Mifos Initiative
>>>>>>
>>>>>> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Collectively Creating a World of 3 Billion Maries | *
>>> http://mifos.org
>>>>>> <http://facebook.com/mifos>  <http://www.twitter.com/mifos>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>
>>>
> 

Mission of Fineract discussion (was Re: Collection Sheet Deprecation ...)

[James Dailey <jd...@apache.org>]

Zayyad, Felix and others -

Indeed.  And all of this was laid out for many years as the strategy.

When I was on the Board of Mifos (the non profit, not the project) we
decided to contribute the MifosX code to the ASF to ensure both:
  a) a broader financial services industry approach, and
  b) a more established foundational home for the core of the project.

There were other reasons, but those remain primary in my mind.

The concept was that Mifos org, which is a Vendor in the ASF parlance,
would continue its Financial Inclusion mission and make sure that the
releases that were coming out of the project would retain the flavor and
intent of the original mifos project (which I started and designed in
2001).  The Mifos AdminUI /CommunityApp open source front end UI gave it a
place in the ecosystem and a way to ensure that alignment.

But, importantly, when we made this decision, we deliberately wanted to see
more financial institutions and fintechs use the backend system, AND
vitally, to help maintain it, with contributions upstream.  That is the
sign of a healthy open source project.

I don't speak for Mifos, but I believe that the vision at Mifos remains
about financial inclusion.   But the vision of Fineract and the mission of
the project is different.  We actually VOTED on this revised statement as a
community, I think five years ago.

So, if you are seeing a difference between Mifos and Fineract, then we have
succeeded in an important goal.  The two are NOT THE SAME.  This is NOT
mifos.

This has been called out in many many Reports to the ASF Board.  See the
wiki pages for those.  The key theme is Vendor neutrality - we do not
privilege any vendor.

And, I would argue that this is the right healthy direction for this
Fineract project.  Microfinance serves an important role in the world, and
continues despite many expecting it to decline or go away.  And, the
mission of the original concept of Mifos was not about Microfinance but
about financial inclusion.  However that happens.

And, fundamentally, I personally remain focused on this - disrupting the
financial industry with open source software that can functionally be the
basis of any bank or fintech globally.  It should lower the cost of
financial inclusion, it should do that via any kind of institution at a
structural level.  Unbanked and underbanked are included.

So, if you are not seeing what you want here, then you can invest more time
and resources to make it more of what you want, but I would also suggest
that the fundamental thing is to have healthy vendors that move the project
forward with lots of "built on top of" functionality that serve a wide
range of financial institutions.  If those components are also open source,
that's ok, but it isn't essential.

** Deprecation process **
Deprecating features is a natural part of any system evolution - we need to
prune the tree from time to time.  If users are demanding those features
then they should speak up - as this thread illustrated - and mechanisms
need to be put in place to "protect them".  Here of course, I am referring
to my last email and testing, to start with.

And, we should do a better job of documenting the product roadmap, and
having clear discussions about deprecating features - Before they are
deprecated.

That starts with you listing your required feature set and documenting it
somewhere perhaps - or relying on the Vendor to speak up for your interests
if you don't.

And, this is a good discussion to have.  Perhaps a community call on this
topic would be welcomed in coming weeks.

James



On Sat, Aug 30, 2025 at 3:13 AM Zayyad A. Said <
zayyad@intrasofttechnologies.com> wrote:

> Hello Fineract community,
>
> I concur with the observations below from Felix.
>
> We have been implementing Mifos since 2011 and we have seen the project
> transitioning from Mifos 2.x to Mifos X and now with the new developments.
> The current focus seems to have shifted from the original mission of
> "Creating a world of 3 Billion Maries where each of the 2 billion poor and
> unbanked has access to the financial resources needed to create a better
> life for themselves and their family."
>
> Group lending has been used for decades and Mifos was originally intended
> for and served the model, the new features being introduced are geared
> towards commercial use. We saw this coming and we had to maintain old fork
> for our MFI clients where we continue to improve based on their needs.
>
> While I have no problem with having new features, I seem not to understand
> why we want to deprecate features of group lending that are still
> being used even if with the minority of users globally yet they do no harm
> to the security and overall performance of the project.
>
> I suggest the leadership to rethink the idea of deprecating collection
> sheet functionality and other key features used in group lending.
>
> Best Regards,
>
> Zayyad A. Said
> Intrasoft Technologies Limited
>
>  *********
>
> *Zayyad A. Said | Chief Executive Officer*
>
> Suite H32, Delamere Flats - Milimani Road, Nairobi.
>
> Cell No.: +254 716 615274 | Skype: *zsaid2011*
>
> Email: zayyad@intrasofttechnologies.com
>
> Schedule Meetings: https://calendly.com/zayyadsaid
>
> <https://calendly.com/zayyadsaid>
>
>
> *" You can achieve what you want if you just help enough others get what
> they want.."*
>
> *Note: Save the environment, please don't print this email unless its
> really necessary for you to do that.*
>
> On Sat, Aug 30, 2025, 11:20 Felix van Hove <fv...@gmx.de.invalid>
> wrote:
>
>> Let me chip in with a more personal perspective.
>>
>> I'm following the development of the Mifos web app and - to a certain
>> extend - Fineract for a couple of months only. My impression from the
>> influx of new features in Fineract is that these are predominantly not
>> for microfinance or small entities, which Mifos once set out to support.
>> In fact - Fineract only states that its efforts are also "including the
>> unbanked and underbanked" - not its focus.
>>
>> I see the deprecation of the Collection Sheet feature in this context.
>> How many new features have been added in the last 12 months that help
>> the "unbanked"? How many have been added that help larger commercial
>> entities and are of no benefit for "the unbanked"?
>>
>> My personal contributions depend solely on Mifos/Fineract supporting
>> efforts for financial inclusion. I regard it as of utter importance
>> respective features don't come as a plugin, but first class citizens.
>>
>> My knowledge regarding fintech is very limited. If I'm wrong or
>> misrespresent things, please tell me.
>>
>> Felix
>>
>> On 29/08/2025 21:01, James Dailey wrote:
>> > Devs -  Are we done with this?
>> >
>> > First, I am NOT ruling out this functionality as important but if no
>> one is
>> > actually demanding it, and no one is willing to maintain it, then it
>> > doesn't get maintained.  That is the nature of open source projects.
>> >
>> > Second, it can happen that an outside vendor (e.g. Mifos) that relies
>> > on this Collection Sheet functionality (or any functionality)
>> inadvertently
>> > drops that from their front end release.  Because the tests at Fineract
>> do
>> > not cover this fully, no one at the Vendor and no one at the Fineract
>> > project will see the functionality fail in a build until a customer or
>> > implementer notices. So, tests are vital for ensuring ongoing
>> > maintainability.
>> >
>> > Third, if this type of thing isn't "naturally" part of the core - then
>> it
>> > will make a lot more sense to have it be an outside extension - in which
>> > case the tests have to be written for the API calls, and a Vendor should
>> > try to get such tests contributed.   I could be wrong about this, I
>> would
>> > be happy to debate where this belongs.
>> >
>> > @Edward Cable <ed...@mifos.org> - what is the assessment and status
>> of
>> > needed functionality and where do you think it should live?
>> >
>> > Thanks,
>> > James
>> >
>> > On Mon, Aug 18, 2025 at 8:21 AM James Dailey <jd...@apache.org>
>> wrote:
>> >
>> >> Sifiso -
>> >>
>> >> Do we have a way to reach the tens of thousands of institutions you
>> >> believe are there?
>> >>
>> >> My belief (just anecdotal information) is that it’s more like a few
>> >> hundred institutions and that out of that number, less than 10% are on
>> a
>> >> recent release.  And, of that smaller number, few, if any, are still
>> using
>> >> Collection Sheets.  I could be wrong - I’d like the data!  I’d say that
>> >> vendors can show up with their data here to help make this vendor
>> neutral
>> >> space more informed.
>> >>
>> >> If group lending and group-member lending and collection sheets are
>> still
>> >> needed, then perhaps those project volunteers can contribute the needed
>> >> updates to keep the functionality useful.
>> >>
>> >> Thanks
>> >> Thanks
>> >>
>> >>
>> >> On Mon, Aug 18, 2025 at 6:48 AM Sifiso Mtetwa <
>> sifiso@skyburgsystems.org>
>> >> wrote:
>> >>
>> >>> Hi guys,
>> >>>
>> >>>
>> >>>
>> >>> This is an interesting topic. I have been wondering why, in general,
>> we
>> >>> seem to be deprecating more and more system functions. The individual
>> >>> collection sheet has served us well over the years and still does, If
>> >>> anything we could improve on its functionality by maybe adding a bulk
>> >>> collection sheet template with little detail compared to a full loan
>> >>> repayment bulk import template.
>> >>>
>> >>>
>> >>>
>> >>> Fineract is used by tens of thousands of organisations throughout the
>> >>> world and most of them are not on this listing and may not have a
>> voice to
>> >>> air their concerns. Maybe we can find a way of exposing this thread to
>> >>> include more voters.
>> >>>
>> >>>
>> >>>
>> >>> Regards,
>> >>>
>> >>>
>> >>>
>> >>> *From:* James Dailey [mailto:jdailey@apache.org]
>> >>> *Sent:* Saturday, 16 August 2025 02:53
>> >>> *To:* dev@fineract.apache.org
>> >>> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
>> >>> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
>> >>>
>> >>>
>> >>>
>> >>> Ed
>> >>>
>> >>>
>> >>>
>> >>> I agree this needs to be discussed but it is important to acknowledge
>> >>> that THIS Fineract listserv is the only official (and required)
>> discussion
>> >>> space.  It is not the intention to bury anything in “an email thread”.
>> >>>
>> >>>
>> >>>
>> >>> When I started up Mifos we spent a lot of time looking at Collection
>> >>> Sheets and designing process flows around them. I fully know that this
>> >>> design direction was important back then in 2002-2006.
>> >>>
>> >>>
>> >>>
>> >>> However if there is no one here asking for them to be retained besides
>> >>> you, then that is a sign that they have perhaps reached an end of
>> their
>> >>> utility.  Or, that the users are not actually here, which is a
>> different
>> >>> problem.
>> >>>
>> >>>
>> >>>
>> >>> So,… If there is a group using collection sheets in production AND
>> they
>> >>> are not on some permanent forked (old) version, then now is the time
>> to
>> >>> speak up.  Here.
>> >>>
>> >>>
>> >>>
>> >>> Generally, I’m fairly certain we can refactor this with an eye toward
>> >>> extracting it from the core.  Repeating the logic in two places makes
>> no
>> >>> sense either.  Collection sheets are kind of assembled from
>> >>> constituent loans and savings.  The balances and due payments should
>> be
>> >>> calculated in the underlying components.
>> >>>
>> >>>
>> >>>
>> >>> As the system gets restructured we need to decide to keep this at
>> all, to
>> >>> keep it in a new place, or as some external concept/plug in. Why
>> wouldn’t
>> >>> we want a separate component?
>> >>>
>> >>>
>> >>>
>> >>> Cheers
>> >>>
>> >>>
>> >>>
>> >>> On Thu, Aug 14, 2025 at 4:16 PM Ed Cable <ed...@mifos.org> wrote:
>> >>>
>> >>> I'll leave the other thread for discussion of the API versioning and
>> >>> refactoring related to Collection Sheet API but wanted to create a
>> separate
>> >>> thread regarding the deprecation of the Collection Sheet.
>> >>>
>> >>>
>> >>>
>> >>> In general, for this and removal of any functionality, it's something
>> >>> that needs to be discussed openly with the community and with a vote
>> and
>> >>> not a decision buried in a mailing list thread.
>> >>>
>> >>>
>> >>>
>> >>> For the collection sheet specifically, more thought has to be given to
>> >>> its deprecation as the centrality and highly coupled nature of the
>> >>> collection sheet is being understated as it isn't merely a report
>> that's to
>> >>> be printed or a form filled out via a mobile application. It's a
>> >>> significant portion of the user interface and highly coupled to many
>> of the
>> >>> microfinance features around groups/centers/meeting scheduling, staff
>> >>> assignment, etc.
>> >>>
>> >>>
>> >>>
>> >>> I do agree that from a UI perspective, the collection sheet and other
>> >>> microfinance-centric functionalities and flows should be viewable
>> based on
>> >>> a configurable setting. As it doesn't lend itself to the optimal user
>> >>> experience for a large portion of current Fineract user base.
>> >>>
>> >>>
>> >>>
>> >>> I also am supportive of a strategy of slimming Fineract to its core
>> >>> services and functionality above core Fineract services and APIs can
>> be
>> >>> extracted out.
>> >>>
>> >>>
>> >>>
>> >>> So I do think we should give thoughtful consideration to what
>> abstracting
>> >>> out the collection sheet and corresponding microfinance functionality
>> would
>> >>> look like and what that effort would entail to abstract it out without
>> >>> adversely impacting the original user base of the software but it's
>> not as
>> >>> simple as deprecating these API.
>> >>>
>> >>>
>> >>>
>> >>> I welcome others' thoughts and inputs as I know even with microfinance
>> >>> itself, the methodology has evolved and group lending and the concept
>> of a
>> >>> collection sheet isn't as central as it once was
>> >>>
>> >>>
>> >>>
>> >>> Thanks,
>> >>>
>> >>>
>> >>>
>> >>> Ed
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> On Wed, Aug 13, 2025 at 6:36 PM Kapil Panchal <
>> >>> kapil.panchal.development@gmail.com> wrote:
>> >>>
>> >>> Hi James,
>> >>>
>> >>>
>> >>>
>> >>> I can proceed by marking this feature as @Deprecated and/or
>> performing a
>> >>> safe refactor to remove the API.
>> >>>
>> >>>
>> >>>
>> >>> For API versioning I agree to Aleksandar Vidakovic's proposal on
>> adapting
>> >>> to the SpringBoot v7/Spring Framework v4. If I may, Aleksandar
>> >>> Vidakovic takes the lead on this project and I can help to support the
>> >>> conversion?
>> >>>
>> >>>
>> >>>
>> >>> Thanks,
>> >>>
>> >>> Kapil
>> >>>
>> >>>
>> >>>
>> >>> On Thu, Aug 14, 2025 at 5:07 AM James Dailey <jd...@apache.org>
>> wrote:
>> >>>
>> >>> Hi Kapil
>> >>>
>> >>>
>> >>>
>> >>> I might suggest looking at this as an opportunity to remove the
>> >>> collection sheet entirely from the Fineract namespace.  It’s a legacy
>> >>> concept I and others designed a long time ago, originally in 2002
>> based on
>> >>> collection sheets we gathered from a dozen countries. It is strongly
>> tied
>> >>> to concepts in microfinance field operations, and especially when
>> there was
>> >>> no data connectivity.
>> >>>
>> >>>
>> >>>
>> >>> It belongs perhaps as a sort of external microservice - data loading
>> via
>> >>> a bulk import could still be enabled.
>> >>>
>> >>> The API versioning is a good idea but needs to be more holistic across
>> >>> the platform I think.
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <ad...@gmail.com>
>> wrote:
>> >>>
>> >>> Hi Kapil,
>> >>>
>> >>> Thank you for raising the concerns below. I’ll need some additional
>> >>> details to fully understand your points:
>> >>>
>> >>> 1.      *Collection Sheet API* – You mentioned it appears
>> non-functional
>> >>> and contains several logical errors.
>> >>>
>> >>> o    If it’s indeed not working, that’s a separate, high-priority
>> >>> discussion.
>> >>>
>> >>> o    Could you clarify which logical errors you were referring to, and
>> >>> what specifically makes you think it’s non-functional?
>> >>>
>> >>> 2.      *Service annotations* – You noted that service methods are not
>> >>> annotated with @Service and that beans are defined manually.
>> >>>
>> >>> o    Are you referring to the
>> >>> CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being
>> defined
>> >>> via configuration?
>> >>>
>> >>> 3.      *Repository wrappers annotated with **@Service* – You
>> mentioned
>> >>> that this mandates full unit test coverage but that they should
>> ideally be
>> >>> annotated with @Component.
>> >>>
>> >>> o    Could you point out the exact classes you had in mind?
>> >>>
>> >>> As for the other points, I agree we can refactor and remove redundant
>> >>> logic—please feel free to suggest specific improvements or start work
>> on
>> >>> them immediately!
>> >>>
>> >>> However, be careful by moving anything into the fineract-core… We are
>> >>> aiming to keep it as small as possible as everything is built on top
>> of
>> >>> this module! If collection sheet are used for loans and savings - for
>> >>> example - than the recommended move is NOT to move this logic into
>> core!
>> >>>
>> >>> Either:
>> >>>
>> >>> - we split the logic into fineract-loan and fineract-savings
>> >>>
>> >>> - Move the logic into a new module
>> >>>
>> >>> - Leave it in fineract-provider for now
>> >>>
>> >>>
>> >>>
>> >>> Shall you have any questions, please let us know!
>> >>>
>> >>> Regards,
>> >>> Adam
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> On 2025. Aug 11., at 12:09, Kapil Panchal <
>> >>> kapil.panchal.development@gmail.com> wrote:
>> >>>
>> >>>
>> >>>
>> >>> Hi Adam,
>> >>>
>> >>> I’m currently working on *FINERACT-2290* and have a few questions
>> before
>> >>> I submit a pull request.
>> >>>
>> >>> The *Collection Sheet API* in its current state appears non-functional
>> >>> and contains several logical errors. It seems there was an earlier
>> attempt
>> >>> to convert from a JSON string request parameter to a class-based
>> request
>> >>> object, but:
>> >>>
>> >>> Certain fields are missing.
>> >>>
>> >>> The serializer is not correctly populating the objects, which causes
>> the
>> >>> conditional checks to be bypassed and results in incorrect (false)
>> >>> responses.
>> >>>
>> >>> This change set is *high risk* because it touches most of the loan and
>> >>> savings product logic. I’ve had to refactor almost all major methods.
>> >>> Extensive integration and end-to-end testing will be required to
>> ensure
>> >>> there are no regressions, especially in edge cases. At present, there
>> are
>> >>> no unit or integration tests for this functionality, and test
>> creation is
>> >>> outside the current ticket scope. I’ve been iterating on this for a
>> while,
>> >>> and only today have I reached a stable state after several
>> experimental and
>> >>> build-breaking attempts.
>> >>>
>> >>> *Key Observations:*
>> >>>
>> >>> Service methods are not annotated with @Service; instead, beans are
>> >>> defined manually.
>> >>>
>> >>> Repository wrappers are annotated with @Service. This mandates full
>> unit
>> >>> test coverage for these methods, but they should ideally be annotated
>> with
>> >>> @Component.
>> >>>
>> >>> I agree with prior discussions on separating bean validation — having
>> a
>> >>> dedicated @Component validation class allows the request object to
>> handle
>> >>> checks independent of database queries.
>> >>>
>> >>> Validation components can also perform database-related validations;
>> >>> these can be injected into service classes for cleaner architecture.
>> >>>
>> >>> Such validation components should be placed in *Fineract-Core* so they
>> >>> are reusable across modules, reducing future refactoring needs.
>> >>>
>> >>> The current design of having commands in *Fineract-Core* and
>> >>> handlers/services/repositories in respective modules is good — it
>> cleanly
>> >>> decouples command definition from execution.
>> >>>
>> >>> There is extensive use of this. in singleton contexts (API, Service,
>> >>> Repository). While not harmful, it’s unnecessary boilerplate.
>> >>>
>> >>> Multiple redundant intermediate DTOs exist where the request DTO
>> itself
>> >>> could be reused for data transfer.
>> >>>
>> >>> I found redundant logic — e.g., a for loop with a break statement that
>> >>> effectively executes only once; this can be simplified.
>> >>>
>> >>> Some JDBC template queries use reserved SQL keywords, causing
>> exceptions.
>> >>> Refactoring these queries resolves the issue and returns proper
>> response
>> >>> objects.
>> >>>
>> >>> *Suggestions:*
>> >>>
>> >>> *Where appropriate, large tickets should be broken into subtasks to
>> >>> manage complexity and reviewability.*
>> >>>
>> >>> It may help to have a dedicated *developer-only Slack channel *for
>> >>> technical discussions. This could complement other community spaces if
>> >>> there’s a need to keep certain conversations more focused.
>> >>>
>> >>> What are your thoughts on the above?
>> >>>
>> >>> Thanks,
>> >>> Kapil
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>>
>> >>> *Ed Cable*
>> >>>
>> >>> President/CEO, Mifos Initiative
>> >>>
>> >>> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>> >>>
>> >>>
>> >>>
>> >>> *Collectively Creating a World of 3 Billion Maries | *
>> http://mifos.org
>> >>> <http://facebook.com/mifos>  <http://www.twitter.com/mifos>
>> >>>
>> >>>
>> >>>
>> >>>
>> >
>>
>>

Re: Collection Sheet Deprecation was [Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)]

["Zayyad A. Said" <za...@intrasofttechnologies.com>]

Hello Fineract community,

I concur with the observations below from Felix.

We have been implementing Mifos since 2011 and we have seen the project
transitioning from Mifos 2.x to Mifos X and now with the new developments.
The current focus seems to have shifted from the original mission of
"Creating a world of 3 Billion Maries where each of the 2 billion poor and
unbanked has access to the financial resources needed to create a better
life for themselves and their family."

Group lending has been used for decades and Mifos was originally intended
for and served the model, the new features being introduced are geared
towards commercial use. We saw this coming and we had to maintain old fork
for our MFI clients where we continue to improve based on their needs.

While I have no problem with having new features, I seem not to understand
why we want to deprecate features of group lending that are still
being used even if with the minority of users globally yet they do no harm
to the security and overall performance of the project.

I suggest the leadership to rethink the idea of deprecating collection
sheet functionality and other key features used in group lending.

Best Regards,

Zayyad A. Said
Intrasoft Technologies Limited

 *********

*Zayyad A. Said | Chief Executive Officer*

Suite H32, Delamere Flats - Milimani Road, Nairobi.

Cell No.: +254 716 615274 | Skype: *zsaid2011*

Email: zayyad@intrasofttechnologies.com

Schedule Meetings: https://calendly.com/zayyadsaid

<https://calendly.com/zayyadsaid>


*" You can achieve what you want if you just help enough others get what
they want.."*

*Note: Save the environment, please don't print this email unless its
really necessary for you to do that.*

On Sat, Aug 30, 2025, 11:20 Felix van Hove <fv...@gmx.de.invalid> wrote:

> Let me chip in with a more personal perspective.
>
> I'm following the development of the Mifos web app and - to a certain
> extend - Fineract for a couple of months only. My impression from the
> influx of new features in Fineract is that these are predominantly not
> for microfinance or small entities, which Mifos once set out to support.
> In fact - Fineract only states that its efforts are also "including the
> unbanked and underbanked" - not its focus.
>
> I see the deprecation of the Collection Sheet feature in this context.
> How many new features have been added in the last 12 months that help
> the "unbanked"? How many have been added that help larger commercial
> entities and are of no benefit for "the unbanked"?
>
> My personal contributions depend solely on Mifos/Fineract supporting
> efforts for financial inclusion. I regard it as of utter importance
> respective features don't come as a plugin, but first class citizens.
>
> My knowledge regarding fintech is very limited. If I'm wrong or
> misrespresent things, please tell me.
>
> Felix
>
> On 29/08/2025 21:01, James Dailey wrote:
> > Devs -  Are we done with this?
> >
> > First, I am NOT ruling out this functionality as important but if no one
> is
> > actually demanding it, and no one is willing to maintain it, then it
> > doesn't get maintained.  That is the nature of open source projects.
> >
> > Second, it can happen that an outside vendor (e.g. Mifos) that relies
> > on this Collection Sheet functionality (or any functionality)
> inadvertently
> > drops that from their front end release.  Because the tests at Fineract
> do
> > not cover this fully, no one at the Vendor and no one at the Fineract
> > project will see the functionality fail in a build until a customer or
> > implementer notices. So, tests are vital for ensuring ongoing
> > maintainability.
> >
> > Third, if this type of thing isn't "naturally" part of the core - then it
> > will make a lot more sense to have it be an outside extension - in which
> > case the tests have to be written for the API calls, and a Vendor should
> > try to get such tests contributed.   I could be wrong about this, I would
> > be happy to debate where this belongs.
> >
> > @Edward Cable <ed...@mifos.org> - what is the assessment and status of
> > needed functionality and where do you think it should live?
> >
> > Thanks,
> > James
> >
> > On Mon, Aug 18, 2025 at 8:21 AM James Dailey <jd...@apache.org> wrote:
> >
> >> Sifiso -
> >>
> >> Do we have a way to reach the tens of thousands of institutions you
> >> believe are there?
> >>
> >> My belief (just anecdotal information) is that it’s more like a few
> >> hundred institutions and that out of that number, less than 10% are on a
> >> recent release.  And, of that smaller number, few, if any, are still
> using
> >> Collection Sheets.  I could be wrong - I’d like the data!  I’d say that
> >> vendors can show up with their data here to help make this vendor
> neutral
> >> space more informed.
> >>
> >> If group lending and group-member lending and collection sheets are
> still
> >> needed, then perhaps those project volunteers can contribute the needed
> >> updates to keep the functionality useful.
> >>
> >> Thanks
> >> Thanks
> >>
> >>
> >> On Mon, Aug 18, 2025 at 6:48 AM Sifiso Mtetwa <
> sifiso@skyburgsystems.org>
> >> wrote:
> >>
> >>> Hi guys,
> >>>
> >>>
> >>>
> >>> This is an interesting topic. I have been wondering why, in general, we
> >>> seem to be deprecating more and more system functions. The individual
> >>> collection sheet has served us well over the years and still does, If
> >>> anything we could improve on its functionality by maybe adding a bulk
> >>> collection sheet template with little detail compared to a full loan
> >>> repayment bulk import template.
> >>>
> >>>
> >>>
> >>> Fineract is used by tens of thousands of organisations throughout the
> >>> world and most of them are not on this listing and may not have a
> voice to
> >>> air their concerns. Maybe we can find a way of exposing this thread to
> >>> include more voters.
> >>>
> >>>
> >>>
> >>> Regards,
> >>>
> >>>
> >>>
> >>> *From:* James Dailey [mailto:jdailey@apache.org]
> >>> *Sent:* Saturday, 16 August 2025 02:53
> >>> *To:* dev@fineract.apache.org
> >>> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
> >>> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
> >>>
> >>>
> >>>
> >>> Ed
> >>>
> >>>
> >>>
> >>> I agree this needs to be discussed but it is important to acknowledge
> >>> that THIS Fineract listserv is the only official (and required)
> discussion
> >>> space.  It is not the intention to bury anything in “an email thread”.
> >>>
> >>>
> >>>
> >>> When I started up Mifos we spent a lot of time looking at Collection
> >>> Sheets and designing process flows around them. I fully know that this
> >>> design direction was important back then in 2002-2006.
> >>>
> >>>
> >>>
> >>> However if there is no one here asking for them to be retained besides
> >>> you, then that is a sign that they have perhaps reached an end of their
> >>> utility.  Or, that the users are not actually here, which is a
> different
> >>> problem.
> >>>
> >>>
> >>>
> >>> So,… If there is a group using collection sheets in production AND they
> >>> are not on some permanent forked (old) version, then now is the time to
> >>> speak up.  Here.
> >>>
> >>>
> >>>
> >>> Generally, I’m fairly certain we can refactor this with an eye toward
> >>> extracting it from the core.  Repeating the logic in two places makes
> no
> >>> sense either.  Collection sheets are kind of assembled from
> >>> constituent loans and savings.  The balances and due payments should be
> >>> calculated in the underlying components.
> >>>
> >>>
> >>>
> >>> As the system gets restructured we need to decide to keep this at all,
> to
> >>> keep it in a new place, or as some external concept/plug in. Why
> wouldn’t
> >>> we want a separate component?
> >>>
> >>>
> >>>
> >>> Cheers
> >>>
> >>>
> >>>
> >>> On Thu, Aug 14, 2025 at 4:16 PM Ed Cable <ed...@mifos.org> wrote:
> >>>
> >>> I'll leave the other thread for discussion of the API versioning and
> >>> refactoring related to Collection Sheet API but wanted to create a
> separate
> >>> thread regarding the deprecation of the Collection Sheet.
> >>>
> >>>
> >>>
> >>> In general, for this and removal of any functionality, it's something
> >>> that needs to be discussed openly with the community and with a vote
> and
> >>> not a decision buried in a mailing list thread.
> >>>
> >>>
> >>>
> >>> For the collection sheet specifically, more thought has to be given to
> >>> its deprecation as the centrality and highly coupled nature of the
> >>> collection sheet is being understated as it isn't merely a report
> that's to
> >>> be printed or a form filled out via a mobile application. It's a
> >>> significant portion of the user interface and highly coupled to many
> of the
> >>> microfinance features around groups/centers/meeting scheduling, staff
> >>> assignment, etc.
> >>>
> >>>
> >>>
> >>> I do agree that from a UI perspective, the collection sheet and other
> >>> microfinance-centric functionalities and flows should be viewable
> based on
> >>> a configurable setting. As it doesn't lend itself to the optimal user
> >>> experience for a large portion of current Fineract user base.
> >>>
> >>>
> >>>
> >>> I also am supportive of a strategy of slimming Fineract to its core
> >>> services and functionality above core Fineract services and APIs can be
> >>> extracted out.
> >>>
> >>>
> >>>
> >>> So I do think we should give thoughtful consideration to what
> abstracting
> >>> out the collection sheet and corresponding microfinance functionality
> would
> >>> look like and what that effort would entail to abstract it out without
> >>> adversely impacting the original user base of the software but it's
> not as
> >>> simple as deprecating these API.
> >>>
> >>>
> >>>
> >>> I welcome others' thoughts and inputs as I know even with microfinance
> >>> itself, the methodology has evolved and group lending and the concept
> of a
> >>> collection sheet isn't as central as it once was
> >>>
> >>>
> >>>
> >>> Thanks,
> >>>
> >>>
> >>>
> >>> Ed
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On Wed, Aug 13, 2025 at 6:36 PM Kapil Panchal <
> >>> kapil.panchal.development@gmail.com> wrote:
> >>>
> >>> Hi James,
> >>>
> >>>
> >>>
> >>> I can proceed by marking this feature as @Deprecated and/or performing
> a
> >>> safe refactor to remove the API.
> >>>
> >>>
> >>>
> >>> For API versioning I agree to Aleksandar Vidakovic's proposal on
> adapting
> >>> to the SpringBoot v7/Spring Framework v4. If I may, Aleksandar
> >>> Vidakovic takes the lead on this project and I can help to support the
> >>> conversion?
> >>>
> >>>
> >>>
> >>> Thanks,
> >>>
> >>> Kapil
> >>>
> >>>
> >>>
> >>> On Thu, Aug 14, 2025 at 5:07 AM James Dailey <jd...@apache.org>
> wrote:
> >>>
> >>> Hi Kapil
> >>>
> >>>
> >>>
> >>> I might suggest looking at this as an opportunity to remove the
> >>> collection sheet entirely from the Fineract namespace.  It’s a legacy
> >>> concept I and others designed a long time ago, originally in 2002
> based on
> >>> collection sheets we gathered from a dozen countries. It is strongly
> tied
> >>> to concepts in microfinance field operations, and especially when
> there was
> >>> no data connectivity.
> >>>
> >>>
> >>>
> >>> It belongs perhaps as a sort of external microservice - data loading
> via
> >>> a bulk import could still be enabled.
> >>>
> >>> The API versioning is a good idea but needs to be more holistic across
> >>> the platform I think.
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <ad...@gmail.com>
> wrote:
> >>>
> >>> Hi Kapil,
> >>>
> >>> Thank you for raising the concerns below. I’ll need some additional
> >>> details to fully understand your points:
> >>>
> >>> 1.      *Collection Sheet API* – You mentioned it appears
> non-functional
> >>> and contains several logical errors.
> >>>
> >>> o    If it’s indeed not working, that’s a separate, high-priority
> >>> discussion.
> >>>
> >>> o    Could you clarify which logical errors you were referring to, and
> >>> what specifically makes you think it’s non-functional?
> >>>
> >>> 2.      *Service annotations* – You noted that service methods are not
> >>> annotated with @Service and that beans are defined manually.
> >>>
> >>> o    Are you referring to the
> >>> CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being defined
> >>> via configuration?
> >>>
> >>> 3.      *Repository wrappers annotated with **@Service* – You mentioned
> >>> that this mandates full unit test coverage but that they should
> ideally be
> >>> annotated with @Component.
> >>>
> >>> o    Could you point out the exact classes you had in mind?
> >>>
> >>> As for the other points, I agree we can refactor and remove redundant
> >>> logic—please feel free to suggest specific improvements or start work
> on
> >>> them immediately!
> >>>
> >>> However, be careful by moving anything into the fineract-core… We are
> >>> aiming to keep it as small as possible as everything is built on top of
> >>> this module! If collection sheet are used for loans and savings - for
> >>> example - than the recommended move is NOT to move this logic into
> core!
> >>>
> >>> Either:
> >>>
> >>> - we split the logic into fineract-loan and fineract-savings
> >>>
> >>> - Move the logic into a new module
> >>>
> >>> - Leave it in fineract-provider for now
> >>>
> >>>
> >>>
> >>> Shall you have any questions, please let us know!
> >>>
> >>> Regards,
> >>> Adam
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On 2025. Aug 11., at 12:09, Kapil Panchal <
> >>> kapil.panchal.development@gmail.com> wrote:
> >>>
> >>>
> >>>
> >>> Hi Adam,
> >>>
> >>> I’m currently working on *FINERACT-2290* and have a few questions
> before
> >>> I submit a pull request.
> >>>
> >>> The *Collection Sheet API* in its current state appears non-functional
> >>> and contains several logical errors. It seems there was an earlier
> attempt
> >>> to convert from a JSON string request parameter to a class-based
> request
> >>> object, but:
> >>>
> >>> Certain fields are missing.
> >>>
> >>> The serializer is not correctly populating the objects, which causes
> the
> >>> conditional checks to be bypassed and results in incorrect (false)
> >>> responses.
> >>>
> >>> This change set is *high risk* because it touches most of the loan and
> >>> savings product logic. I’ve had to refactor almost all major methods.
> >>> Extensive integration and end-to-end testing will be required to ensure
> >>> there are no regressions, especially in edge cases. At present, there
> are
> >>> no unit or integration tests for this functionality, and test creation
> is
> >>> outside the current ticket scope. I’ve been iterating on this for a
> while,
> >>> and only today have I reached a stable state after several
> experimental and
> >>> build-breaking attempts.
> >>>
> >>> *Key Observations:*
> >>>
> >>> Service methods are not annotated with @Service; instead, beans are
> >>> defined manually.
> >>>
> >>> Repository wrappers are annotated with @Service. This mandates full
> unit
> >>> test coverage for these methods, but they should ideally be annotated
> with
> >>> @Component.
> >>>
> >>> I agree with prior discussions on separating bean validation — having a
> >>> dedicated @Component validation class allows the request object to
> handle
> >>> checks independent of database queries.
> >>>
> >>> Validation components can also perform database-related validations;
> >>> these can be injected into service classes for cleaner architecture.
> >>>
> >>> Such validation components should be placed in *Fineract-Core* so they
> >>> are reusable across modules, reducing future refactoring needs.
> >>>
> >>> The current design of having commands in *Fineract-Core* and
> >>> handlers/services/repositories in respective modules is good — it
> cleanly
> >>> decouples command definition from execution.
> >>>
> >>> There is extensive use of this. in singleton contexts (API, Service,
> >>> Repository). While not harmful, it’s unnecessary boilerplate.
> >>>
> >>> Multiple redundant intermediate DTOs exist where the request DTO itself
> >>> could be reused for data transfer.
> >>>
> >>> I found redundant logic — e.g., a for loop with a break statement that
> >>> effectively executes only once; this can be simplified.
> >>>
> >>> Some JDBC template queries use reserved SQL keywords, causing
> exceptions.
> >>> Refactoring these queries resolves the issue and returns proper
> response
> >>> objects.
> >>>
> >>> *Suggestions:*
> >>>
> >>> *Where appropriate, large tickets should be broken into subtasks to
> >>> manage complexity and reviewability.*
> >>>
> >>> It may help to have a dedicated *developer-only Slack channel *for
> >>> technical discussions. This could complement other community spaces if
> >>> there’s a need to keep certain conversations more focused.
> >>>
> >>> What are your thoughts on the above?
> >>>
> >>> Thanks,
> >>> Kapil
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>>
> >>> *Ed Cable*
> >>>
> >>> President/CEO, Mifos Initiative
> >>>
> >>> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
> >>>
> >>>
> >>>
> >>> *Collectively Creating a World of 3 Billion Maries | *http://mifos.org
> >>> <http://facebook.com/mifos>  <http://www.twitter.com/mifos>
> >>>
> >>>
> >>>
> >>>
> >
>
>

Re: Collection Sheet Deprecation was [Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)]

[Felix van Hove <fv...@gmx.de.INVALID>]

Let me chip in with a more personal perspective.

I'm following the development of the Mifos web app and - to a certain 
extend - Fineract for a couple of months only. My impression from the 
influx of new features in Fineract is that these are predominantly not 
for microfinance or small entities, which Mifos once set out to support. 
In fact - Fineract only states that its efforts are also "including the 
unbanked and underbanked" - not its focus.

I see the deprecation of the Collection Sheet feature in this context. 
How many new features have been added in the last 12 months that help 
the "unbanked"? How many have been added that help larger commercial 
entities and are of no benefit for "the unbanked"?

My personal contributions depend solely on Mifos/Fineract supporting 
efforts for financial inclusion. I regard it as of utter importance 
respective features don't come as a plugin, but first class citizens.

My knowledge regarding fintech is very limited. If I'm wrong or 
misrespresent things, please tell me.

Felix

On 29/08/2025 21:01, James Dailey wrote:
> Devs -  Are we done with this?
> 
> First, I am NOT ruling out this functionality as important but if no one is
> actually demanding it, and no one is willing to maintain it, then it
> doesn't get maintained.  That is the nature of open source projects.
> 
> Second, it can happen that an outside vendor (e.g. Mifos) that relies
> on this Collection Sheet functionality (or any functionality) inadvertently
> drops that from their front end release.  Because the tests at Fineract do
> not cover this fully, no one at the Vendor and no one at the Fineract
> project will see the functionality fail in a build until a customer or
> implementer notices. So, tests are vital for ensuring ongoing
> maintainability.
> 
> Third, if this type of thing isn't "naturally" part of the core - then it
> will make a lot more sense to have it be an outside extension - in which
> case the tests have to be written for the API calls, and a Vendor should
> try to get such tests contributed.   I could be wrong about this, I would
> be happy to debate where this belongs.
> 
> @Edward Cable <ed...@mifos.org> - what is the assessment and status of
> needed functionality and where do you think it should live?
> 
> Thanks,
> James
> 
> On Mon, Aug 18, 2025 at 8:21 AM James Dailey <jd...@apache.org> wrote:
> 
>> Sifiso -
>>
>> Do we have a way to reach the tens of thousands of institutions you
>> believe are there?
>>
>> My belief (just anecdotal information) is that it’s more like a few
>> hundred institutions and that out of that number, less than 10% are on a
>> recent release.  And, of that smaller number, few, if any, are still using
>> Collection Sheets.  I could be wrong - I’d like the data!  I’d say that
>> vendors can show up with their data here to help make this vendor neutral
>> space more informed.
>>
>> If group lending and group-member lending and collection sheets are still
>> needed, then perhaps those project volunteers can contribute the needed
>> updates to keep the functionality useful.
>>
>> Thanks
>> Thanks
>>
>>
>> On Mon, Aug 18, 2025 at 6:48 AM Sifiso Mtetwa <si...@skyburgsystems.org>
>> wrote:
>>
>>> Hi guys,
>>>
>>>
>>>
>>> This is an interesting topic. I have been wondering why, in general, we
>>> seem to be deprecating more and more system functions. The individual
>>> collection sheet has served us well over the years and still does, If
>>> anything we could improve on its functionality by maybe adding a bulk
>>> collection sheet template with little detail compared to a full loan
>>> repayment bulk import template.
>>>
>>>
>>>
>>> Fineract is used by tens of thousands of organisations throughout the
>>> world and most of them are not on this listing and may not have a voice to
>>> air their concerns. Maybe we can find a way of exposing this thread to
>>> include more voters.
>>>
>>>
>>>
>>> Regards,
>>>
>>>
>>>
>>> *From:* James Dailey [mailto:jdailey@apache.org]
>>> *Sent:* Saturday, 16 August 2025 02:53
>>> *To:* dev@fineract.apache.org
>>> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
>>> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
>>>
>>>
>>>
>>> Ed
>>>
>>>
>>>
>>> I agree this needs to be discussed but it is important to acknowledge
>>> that THIS Fineract listserv is the only official (and required) discussion
>>> space.  It is not the intention to bury anything in “an email thread”.
>>>
>>>
>>>
>>> When I started up Mifos we spent a lot of time looking at Collection
>>> Sheets and designing process flows around them. I fully know that this
>>> design direction was important back then in 2002-2006.
>>>
>>>
>>>
>>> However if there is no one here asking for them to be retained besides
>>> you, then that is a sign that they have perhaps reached an end of their
>>> utility.  Or, that the users are not actually here, which is a different
>>> problem.
>>>
>>>
>>>
>>> So,… If there is a group using collection sheets in production AND they
>>> are not on some permanent forked (old) version, then now is the time to
>>> speak up.  Here.
>>>
>>>
>>>
>>> Generally, I’m fairly certain we can refactor this with an eye toward
>>> extracting it from the core.  Repeating the logic in two places makes no
>>> sense either.  Collection sheets are kind of assembled from
>>> constituent loans and savings.  The balances and due payments should be
>>> calculated in the underlying components.
>>>
>>>
>>>
>>> As the system gets restructured we need to decide to keep this at all, to
>>> keep it in a new place, or as some external concept/plug in. Why wouldn’t
>>> we want a separate component?
>>>
>>>
>>>
>>> Cheers
>>>
>>>
>>>
>>> On Thu, Aug 14, 2025 at 4:16 PM Ed Cable <ed...@mifos.org> wrote:
>>>
>>> I'll leave the other thread for discussion of the API versioning and
>>> refactoring related to Collection Sheet API but wanted to create a separate
>>> thread regarding the deprecation of the Collection Sheet.
>>>
>>>
>>>
>>> In general, for this and removal of any functionality, it's something
>>> that needs to be discussed openly with the community and with a vote and
>>> not a decision buried in a mailing list thread.
>>>
>>>
>>>
>>> For the collection sheet specifically, more thought has to be given to
>>> its deprecation as the centrality and highly coupled nature of the
>>> collection sheet is being understated as it isn't merely a report that's to
>>> be printed or a form filled out via a mobile application. It's a
>>> significant portion of the user interface and highly coupled to many of the
>>> microfinance features around groups/centers/meeting scheduling, staff
>>> assignment, etc.
>>>
>>>
>>>
>>> I do agree that from a UI perspective, the collection sheet and other
>>> microfinance-centric functionalities and flows should be viewable based on
>>> a configurable setting. As it doesn't lend itself to the optimal user
>>> experience for a large portion of current Fineract user base.
>>>
>>>
>>>
>>> I also am supportive of a strategy of slimming Fineract to its core
>>> services and functionality above core Fineract services and APIs can be
>>> extracted out.
>>>
>>>
>>>
>>> So I do think we should give thoughtful consideration to what abstracting
>>> out the collection sheet and corresponding microfinance functionality would
>>> look like and what that effort would entail to abstract it out without
>>> adversely impacting the original user base of the software but it's not as
>>> simple as deprecating these API.
>>>
>>>
>>>
>>> I welcome others' thoughts and inputs as I know even with microfinance
>>> itself, the methodology has evolved and group lending and the concept of a
>>> collection sheet isn't as central as it once was
>>>
>>>
>>>
>>> Thanks,
>>>
>>>
>>>
>>> Ed
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Wed, Aug 13, 2025 at 6:36 PM Kapil Panchal <
>>> kapil.panchal.development@gmail.com> wrote:
>>>
>>> Hi James,
>>>
>>>
>>>
>>> I can proceed by marking this feature as @Deprecated and/or performing a
>>> safe refactor to remove the API.
>>>
>>>
>>>
>>> For API versioning I agree to Aleksandar Vidakovic's proposal on adapting
>>> to the SpringBoot v7/Spring Framework v4. If I may, Aleksandar
>>> Vidakovic takes the lead on this project and I can help to support the
>>> conversion?
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Kapil
>>>
>>>
>>>
>>> On Thu, Aug 14, 2025 at 5:07 AM James Dailey <jd...@apache.org> wrote:
>>>
>>> Hi Kapil
>>>
>>>
>>>
>>> I might suggest looking at this as an opportunity to remove the
>>> collection sheet entirely from the Fineract namespace.  It’s a legacy
>>> concept I and others designed a long time ago, originally in 2002 based on
>>> collection sheets we gathered from a dozen countries. It is strongly tied
>>> to concepts in microfinance field operations, and especially when there was
>>> no data connectivity.
>>>
>>>
>>>
>>> It belongs perhaps as a sort of external microservice - data loading via
>>> a bulk import could still be enabled.
>>>
>>> The API versioning is a good idea but needs to be more holistic across
>>> the platform I think.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <ad...@gmail.com> wrote:
>>>
>>> Hi Kapil,
>>>
>>> Thank you for raising the concerns below. I’ll need some additional
>>> details to fully understand your points:
>>>
>>> 1.      *Collection Sheet API* – You mentioned it appears non-functional
>>> and contains several logical errors.
>>>
>>> o    If it’s indeed not working, that’s a separate, high-priority
>>> discussion.
>>>
>>> o    Could you clarify which logical errors you were referring to, and
>>> what specifically makes you think it’s non-functional?
>>>
>>> 2.      *Service annotations* – You noted that service methods are not
>>> annotated with @Service and that beans are defined manually.
>>>
>>> o    Are you referring to the
>>> CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being defined
>>> via configuration?
>>>
>>> 3.      *Repository wrappers annotated with **@Service* – You mentioned
>>> that this mandates full unit test coverage but that they should ideally be
>>> annotated with @Component.
>>>
>>> o    Could you point out the exact classes you had in mind?
>>>
>>> As for the other points, I agree we can refactor and remove redundant
>>> logic—please feel free to suggest specific improvements or start work on
>>> them immediately!
>>>
>>> However, be careful by moving anything into the fineract-core… We are
>>> aiming to keep it as small as possible as everything is built on top of
>>> this module! If collection sheet are used for loans and savings - for
>>> example - than the recommended move is NOT to move this logic into core!
>>>
>>> Either:
>>>
>>> - we split the logic into fineract-loan and fineract-savings
>>>
>>> - Move the logic into a new module
>>>
>>> - Leave it in fineract-provider for now
>>>
>>>
>>>
>>> Shall you have any questions, please let us know!
>>>
>>> Regards,
>>> Adam
>>>
>>>
>>>
>>>
>>>
>>> On 2025. Aug 11., at 12:09, Kapil Panchal <
>>> kapil.panchal.development@gmail.com> wrote:
>>>
>>>
>>>
>>> Hi Adam,
>>>
>>> I’m currently working on *FINERACT-2290* and have a few questions before
>>> I submit a pull request.
>>>
>>> The *Collection Sheet API* in its current state appears non-functional
>>> and contains several logical errors. It seems there was an earlier attempt
>>> to convert from a JSON string request parameter to a class-based request
>>> object, but:
>>>
>>> Certain fields are missing.
>>>
>>> The serializer is not correctly populating the objects, which causes the
>>> conditional checks to be bypassed and results in incorrect (false)
>>> responses.
>>>
>>> This change set is *high risk* because it touches most of the loan and
>>> savings product logic. I’ve had to refactor almost all major methods.
>>> Extensive integration and end-to-end testing will be required to ensure
>>> there are no regressions, especially in edge cases. At present, there are
>>> no unit or integration tests for this functionality, and test creation is
>>> outside the current ticket scope. I’ve been iterating on this for a while,
>>> and only today have I reached a stable state after several experimental and
>>> build-breaking attempts.
>>>
>>> *Key Observations:*
>>>
>>> Service methods are not annotated with @Service; instead, beans are
>>> defined manually.
>>>
>>> Repository wrappers are annotated with @Service. This mandates full unit
>>> test coverage for these methods, but they should ideally be annotated with
>>> @Component.
>>>
>>> I agree with prior discussions on separating bean validation — having a
>>> dedicated @Component validation class allows the request object to handle
>>> checks independent of database queries.
>>>
>>> Validation components can also perform database-related validations;
>>> these can be injected into service classes for cleaner architecture.
>>>
>>> Such validation components should be placed in *Fineract-Core* so they
>>> are reusable across modules, reducing future refactoring needs.
>>>
>>> The current design of having commands in *Fineract-Core* and
>>> handlers/services/repositories in respective modules is good — it cleanly
>>> decouples command definition from execution.
>>>
>>> There is extensive use of this. in singleton contexts (API, Service,
>>> Repository). While not harmful, it’s unnecessary boilerplate.
>>>
>>> Multiple redundant intermediate DTOs exist where the request DTO itself
>>> could be reused for data transfer.
>>>
>>> I found redundant logic — e.g., a for loop with a break statement that
>>> effectively executes only once; this can be simplified.
>>>
>>> Some JDBC template queries use reserved SQL keywords, causing exceptions.
>>> Refactoring these queries resolves the issue and returns proper response
>>> objects.
>>>
>>> *Suggestions:*
>>>
>>> *Where appropriate, large tickets should be broken into subtasks to
>>> manage complexity and reviewability.*
>>>
>>> It may help to have a dedicated *developer-only Slack channel *for
>>> technical discussions. This could complement other community spaces if
>>> there’s a need to keep certain conversations more focused.
>>>
>>> What are your thoughts on the above?
>>>
>>> Thanks,
>>> Kapil
>>>
>>>
>>>
>>>
>>>
>>>
>>> --
>>>
>>> *Ed Cable*
>>>
>>> President/CEO, Mifos Initiative
>>>
>>> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>>>
>>>
>>>
>>> *Collectively Creating a World of 3 Billion Maries | *http://mifos.org
>>> <http://facebook.com/mifos>  <http://www.twitter.com/mifos>
>>>
>>>
>>>
>>>
> 

Re: Collection Sheet Deprecation was [Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)]

[Kapil Panchal <ka...@gmail.com>]

All,

I have created a Collection Sheet Jira Ticket
https://issues.apache.org/jira/browse/FINERACT-2360 as discussed.

Thanks,
Kapil

On Tue, Sep 2, 2025 at 4:53 PM Kapil Panchal <
kapil.panchal.development@gmail.com> wrote:

> Victor, All,
>
> I can maintain the Collection Sheet efforts for the Project. I would need
> all the necessary clearances and permissions for the same. As regards the
> functional and non-functional requirements of the project, a separate tool
> is needed eg. IBM Rational Doors or similar. The requirements also need to
> be revisioned similar to the product revisions/versions that we have. Any
> change in the project has to be communicated so that impact analyses can be
> done in terms of code changes, code coverage and also the documentation
> changes. All IP generated will be documented as a reference.
>
> I would also need a peer reviewer assigned who can review the pull
> requests? I will keep the stakeholders posted.
>
> Thanks,
> Kapil
>
> On Tue, Sep 2, 2025 at 9:46 AM VICTOR MANUEL ROMERO RODRIGUEZ <
> victor.romero@fintecheando.mx> wrote:
>
>> Kapil/Felix,
>>
>> Is it possible to show/write a description from a user/functional
>> perspective?
>>
>> Kapil
>>
>> Your description is useful but for the non functional
>> requirement (refactoring).
>>
>> Regards
>>
>> El lun, 1 sept 2025 a las 21:53, Kapil Panchal (<
>> kapil.panchal.development@gmail.com>) escribió:
>>
>>> The current Collection Sheet efforts started as an attempt to raise a
>>> pull request to the Jira Ticket
>>> https://issues.apache.org/jira/browse/FINERACT-2290, this ticket aims
>>> at solving the technical debt
>>> https://issues.apache.org/jira/browse/FINERACT-2169, In other words, it
>>> replaces the existing (old/archaic) logic processing (CQRS, Serialization
>>> etc) with the new and improved/latest solution which removes much of the
>>> boilerplate and aims at making the code lean such that all the testing and
>>> related infrastructure needed to manage the quality of the code and
>>> deliverables are minimum, it does that by leveraging the framework and the
>>> supported libraries itself. The framework is fully tested and baselined, if
>>> the code that we write falls inline with framework coding/logic paradigm
>>> then most of the testing need not be done as it has already been tested,
>>> this is a lot akin to testing the database with ACID. None of our code
>>> tests the database/framework/webserver/libraries as they are fully tested
>>> and these are the first principles of a good software design, minimal code
>>> maximal output.
>>>
>>> With the above in mind and the instructions in the card itself which
>>> assumes that the command processing logic is to be changed and not the
>>> business logic itself.
>>>
>>> Cutting the story short, I have attached herewith a UML Class diagram to
>>> elicit why the existing CollectionSheet is non functional. The API accepts
>>> 2 parameters as stated below,
>>>
>>> 1) GenerateCollectionSheet,
>>> 2) SaveCollectionSheet.
>>>
>>> Note1: The first parameter 1) Generate Collection Sheet makes a call to
>>> the Service which inturn has a JDBC logic where mysql reserved keywords are
>>> used, this always returns an exception. (This could have been a recent
>>> mysql release and previously this query might have been working? I am using
>>> the mysql version as stated in the .md documentation).
>>>
>>> Note2: The second parameter 2) A CQRS command that is audited, has three
>>> private methods a) updateBulkRepayments, b) updateBulkDisbursals, c)
>>> updateBulkMandatorySavingsDuePayments have errors in extracting the json
>>> payload one of the methods is mentioned in the UML. There are other errors
>>> too that make this Collection Sheet take code paths which do not reflect
>>> the true metric.
>>>
>>> [image: Model_CollectionSheet_v1_Class_Diagram.PNG]
>>> An excerpt from the code: The bold code is where the extraction of JSON
>>> objects if not done correctly will bypass most of the logic and return
>>> flawed response objects. Here it is the case. Please refer to the chain
>>> emails where the correct logic to extract a variable from the json payload
>>> is highlighted.
>>>
>>> public Collection<SavingsAccountTransactionDTO>
>>> assembleBulkMandatorySavingsAccountTransactionDTOs(final JsonCommand
>>> command,
>>>
>>>         final PaymentDetail paymentDetail) {
>>>     final String json = command.json();
>>>     if (StringUtils.isBlank(json)) {
>>>         throw new InvalidJsonException();
>>>     }
>>>     final JsonElement element = this.fromApiJsonHelper.parse(json);
>>>     final Collection<SavingsAccountTransactionDTO>
>>> savingsAccountTransactions = new ArrayList<>();
>>>     final LocalDate transactionDate =
>>> this.fromApiJsonHelper.extractLocalDateNamed(transactionDateParamName,
>>> element);
>>>     final String dateFormat =
>>> this.fromApiJsonHelper.extractDateFormatParameter(element.getAsJsonObject());
>>>     final JsonObject topLevelJsonElement = element.getAsJsonObject();
>>>     final Locale locale =
>>> this.fromApiJsonHelper.extractLocaleParameter(topLevelJsonElement);
>>>     final DateTimeFormatter formatter =
>>> DateTimeFormatter.ofPattern(dateFormat).withLocale(locale);
>>>
>>>
>>>
>>>
>>>
>>>
>>> * if (element.isJsonObject()) {        if
>>> (topLevelJsonElement.has(bulkSavingsDueTransactionsParamName)
>>>   &&
>>> topLevelJsonElement.get(bulkSavingsDueTransactionsParamName).isJsonArray())
>>> {            final JsonArray array =
>>> topLevelJsonElement.get(bulkSavingsDueTransactionsParamName).getAsJsonArray();
>>>           for (int i = 0; i < array.size(); i++) {*
>>>
>>>
>>> "If you can raise a JIRA for each of the logical errors (and ideally
>>> provide add a test case into the automated tests to demonstrate each), then
>>> let's first fix these errors. After the API works as expected, we can then
>>> proceed to refactor it."
>>>
>>>
>>> I think this effort can be broken down to 3-4 Jira tickets for it to be
>>> fully functional. Assuming that we will be using the same command
>>> processing logic as we have been using and not the latest one as per
>>> https://issues.apache.org/jira/browse/FINERACT-2169.
>>>
>>> 1) Jira Ticket 1 - Work on getting the Generate Collection Sheet up and
>>> running, including unit tests and integration tests.
>>> 2) Jira Ticket 2 - Work on getting the Save Collection Sheet up and
>>> running, excluding unit tests and/or integration tests (some refactoring).
>>> 3) Jira Ticket 3 - Work on getting the Save Collection Sheet unit tests.
>>> 4) Jira Ticket 4 - Work on getting the Save Collection Sheet integration
>>> tests.
>>>
>>> The release to production of tickets 2 - 4 can be done when all are
>>> approved? Please let me know so that I can start working on those ASAP.
>>>
>>> Thanks,
>>> Kapil
>>>
>>> On Tue, Sep 2, 2025 at 4:48 AM Petri Tuomola <pe...@gmail.com>
>>> wrote:
>>>
>>>> Thanks Kapil - but this JIRA / PRs seem to be about refactoring the
>>>> Collection Sheet API to the new command processing pattern, not about
>>>> fixing defects in the API.
>>>>
>>>> Before we start any refactoring, we should make sure we're starting
>>>> from a known good state and that there's sufficient unit testing coverage
>>>> to test the functionality, so we know if we've broken something during
>>>> refactoring.
>>>>
>>>> To get to this point, can you share what your comment "The *Collection
>>>> Sheet API* in its current state appears non-functional and contains
>>>> several logical errors" refers to?
>>>>
>>>> If you can raise a JIRA for each of the logical errors (and ideally
>>>> provide add a test case into the automated tests to demonstrate each), then
>>>> let's first fix these errors. After the API works as expected, we can then
>>>> proceed to refactor it.
>>>>
>>>> Regards
>>>> Petri
>>>>
>>>>
>>>>
>>>> ------------------------------
>>>> *From:* Kapil Panchal <ka...@gmail.com>
>>>> *Sent:* Monday, September 1, 2025 7:42 PM
>>>> *To:* dev@fineract.apache.org <de...@fineract.apache.org>
>>>> *Cc:* Edward Cable <ed...@mifos.org>
>>>> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
>>>> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
>>>>
>>>> Hi Petri,
>>>>
>>>> Please take a look at the Jira Ticket
>>>> https://issues.apache.org/jira/browse/FINERACT-2290. I had released a
>>>> Pull Request https://github.com/apache/fineract/pull/4943.
>>>>
>>>> Thanks,
>>>> Kapil
>>>>
>>>> On Mon, Sep 1, 2025 at 7:27 AM Petri Tuomola <pe...@gmail.com>
>>>> wrote:
>>>>
>>>> Hi
>>>>
>>>> I'm happy to help fix issues with Collection Sheet API.
>>>>
>>>> If I look at the emails that started all this, I can only see this
>>>> comment:
>>>>
>>>> "The *Collection Sheet API* in its current state appears
>>>> non-functional and contains several logical errors"
>>>>
>>>>
>>>> I think it's fair to expect a bit more detailed bug report than that.
>>>>
>>>> Is there any more details on this issue: how is it non-functional, and
>>>> what are the logical errors?
>>>>
>>>>
>>>> Who can provide the list of JIRAs that need to be fixed? I can then
>>>> have a look this weekend.
>>>>
>>>>
>>>> Regards
>>>>
>>>> Petri
>>>>
>>>>
>>>>
>>>> ------------------------------
>>>> *From:* James Dailey <jd...@apache.org>
>>>> *Sent:* Saturday, August 30, 2025 3:01 AM
>>>> *To:* dev@fineract.apache.org <de...@fineract.apache.org>; Edward Cable <
>>>> edcable@mifos.org>
>>>> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
>>>> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
>>>>
>>>> Devs -  Are we done with this?
>>>>
>>>> First, I am NOT ruling out this functionality as important but if no
>>>> one is actually demanding it, and no one is willing to maintain it, then it
>>>> doesn't get maintained.  That is the nature of open source projects.
>>>>
>>>> Second, it can happen that an outside vendor (e.g. Mifos) that relies
>>>> on this Collection Sheet functionality (or any functionality) inadvertently
>>>> drops that from their front end release.  Because the tests at Fineract do
>>>> not cover this fully, no one at the Vendor and no one at the Fineract
>>>> project will see the functionality fail in a build until a customer or
>>>> implementer notices. So, tests are vital for ensuring ongoing
>>>> maintainability.
>>>>
>>>> Third, if this type of thing isn't "naturally" part of the core - then
>>>> it will make a lot more sense to have it be an outside extension - in which
>>>> case the tests have to be written for the API calls, and a Vendor should
>>>> try to get such tests contributed.   I could be wrong about this, I would
>>>> be happy to debate where this belongs.
>>>>
>>>> @Edward Cable <ed...@mifos.org> - what is the assessment and status
>>>> of needed functionality and where do you think it should live?
>>>>
>>>> Thanks,
>>>> James
>>>>
>>>> On Mon, Aug 18, 2025 at 8:21 AM James Dailey <jd...@apache.org>
>>>> wrote:
>>>>
>>>> Sifiso -
>>>>
>>>> Do we have a way to reach the tens of thousands of institutions you
>>>> believe are there?
>>>>
>>>> My belief (just anecdotal information) is that it’s more like a few
>>>> hundred institutions and that out of that number, less than 10% are on a
>>>> recent release.  And, of that smaller number, few, if any, are still using
>>>> Collection Sheets.  I could be wrong - I’d like the data!  I’d say that
>>>> vendors can show up with their data here to help make this vendor neutral
>>>> space more informed.
>>>>
>>>> If group lending and group-member lending and collection sheets are
>>>> still needed, then perhaps those project volunteers can contribute the
>>>> needed updates to keep the functionality useful.
>>>>
>>>> Thanks
>>>> Thanks
>>>>
>>>>
>>>> On Mon, Aug 18, 2025 at 6:48 AM Sifiso Mtetwa <
>>>> sifiso@skyburgsystems.org> wrote:
>>>>
>>>> Hi guys,
>>>>
>>>>
>>>>
>>>> This is an interesting topic. I have been wondering why, in general, we
>>>> seem to be deprecating more and more system functions. The individual
>>>> collection sheet has served us well over the years and still does, If
>>>> anything we could improve on its functionality by maybe adding a bulk
>>>> collection sheet template with little detail compared to a full loan
>>>> repayment bulk import template.
>>>>
>>>>
>>>>
>>>> Fineract is used by tens of thousands of organisations throughout the
>>>> world and most of them are not on this listing and may not have a voice to
>>>> air their concerns. Maybe we can find a way of exposing this thread to
>>>> include more voters.
>>>>
>>>>
>>>>
>>>> Regards,
>>>>
>>>>
>>>>
>>>> *From:* James Dailey [mailto:jdailey@apache.org]
>>>> *Sent:* Saturday, 16 August 2025 02:53
>>>> *To:* dev@fineract.apache.org
>>>> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
>>>> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
>>>>
>>>>
>>>>
>>>> Ed
>>>>
>>>>
>>>>
>>>> I agree this needs to be discussed but it is important to acknowledge
>>>> that THIS Fineract listserv is the only official (and required) discussion
>>>> space.  It is not the intention to bury anything in “an email thread”.
>>>>
>>>>
>>>>
>>>> When I started up Mifos we spent a lot of time looking at Collection
>>>> Sheets and designing process flows around them. I fully know that this
>>>> design direction was important back then in 2002-2006.
>>>>
>>>>
>>>>
>>>> However if there is no one here asking for them to be retained besides
>>>> you, then that is a sign that they have perhaps reached an end of
>>>> their utility.  Or, that the users are not actually here, which is a
>>>> different problem.
>>>>
>>>>
>>>>
>>>> So,… If there is a group using collection sheets in production AND they
>>>> are not on some permanent forked (old) version, then now is the time to
>>>> speak up.  Here.
>>>>
>>>>
>>>>
>>>> Generally, I’m fairly certain we can refactor this with an eye toward
>>>> extracting it from the core.  Repeating the logic in two places makes no
>>>> sense either.  Collection sheets are kind of assembled from
>>>> constituent loans and savings.  The balances and due payments should be
>>>> calculated in the underlying components.
>>>>
>>>>
>>>>
>>>> As the system gets restructured we need to decide to keep this at all,
>>>> to keep it in a new place, or as some external concept/plug in. Why
>>>> wouldn’t we want a separate component?
>>>>
>>>>
>>>>
>>>> Cheers
>>>>
>>>>
>>>>
>>>> On Thu, Aug 14, 2025 at 4:16 PM Ed Cable <ed...@mifos.org> wrote:
>>>>
>>>> I'll leave the other thread for discussion of the API versioning and
>>>> refactoring related to Collection Sheet API but wanted to create a separate
>>>> thread regarding the deprecation of the Collection Sheet.
>>>>
>>>>
>>>>
>>>> In general, for this and removal of any functionality, it's something
>>>> that needs to be discussed openly with the community and with a vote and
>>>> not a decision buried in a mailing list thread.
>>>>
>>>>
>>>>
>>>> For the collection sheet specifically, more thought has to be given to
>>>> its deprecation as the centrality and highly coupled nature of the
>>>> collection sheet is being understated as it isn't merely a report that's to
>>>> be printed or a form filled out via a mobile application. It's a
>>>> significant portion of the user interface and highly coupled to many of the
>>>> microfinance features around groups/centers/meeting scheduling, staff
>>>> assignment, etc.
>>>>
>>>>
>>>>
>>>> I do agree that from a UI perspective, the collection sheet and other
>>>> microfinance-centric functionalities and flows should be viewable based on
>>>> a configurable setting. As it doesn't lend itself to the optimal user
>>>> experience for a large portion of current Fineract user base.
>>>>
>>>>
>>>>
>>>> I also am supportive of a strategy of slimming Fineract to its core
>>>> services and functionality above core Fineract services and APIs can be
>>>> extracted out.
>>>>
>>>>
>>>>
>>>> So I do think we should give thoughtful consideration to what
>>>> abstracting out the collection sheet and corresponding microfinance
>>>> functionality would look like and what that effort would entail to abstract
>>>> it out without adversely impacting the original user base of the software
>>>> but it's not as simple as deprecating these API.
>>>>
>>>>
>>>>
>>>> I welcome others' thoughts and inputs as I know even with microfinance
>>>> itself, the methodology has evolved and group lending and the concept of a
>>>> collection sheet isn't as central as it once was
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>>
>>>>
>>>> Ed
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, Aug 13, 2025 at 6:36 PM Kapil Panchal <
>>>> kapil.panchal.development@gmail.com> wrote:
>>>>
>>>> Hi James,
>>>>
>>>>
>>>>
>>>> I can proceed by marking this feature as @Deprecated and/or performing
>>>> a safe refactor to remove the API.
>>>>
>>>>
>>>>
>>>> For API versioning I agree to Aleksandar Vidakovic's proposal on
>>>> adapting to the SpringBoot v7/Spring Framework v4. If I may, Aleksandar
>>>> Vidakovic takes the lead on this project and I can help to support the
>>>> conversion?
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Kapil
>>>>
>>>>
>>>>
>>>> On Thu, Aug 14, 2025 at 5:07 AM James Dailey <jd...@apache.org>
>>>> wrote:
>>>>
>>>> Hi Kapil
>>>>
>>>>
>>>>
>>>> I might suggest looking at this as an opportunity to remove the
>>>> collection sheet entirely from the Fineract namespace.  It’s a legacy
>>>> concept I and others designed a long time ago, originally in 2002 based on
>>>> collection sheets we gathered from a dozen countries. It is strongly tied
>>>> to concepts in microfinance field operations, and especially when there was
>>>> no data connectivity.
>>>>
>>>>
>>>>
>>>> It belongs perhaps as a sort of external microservice - data loading
>>>> via a bulk import could still be enabled.
>>>>
>>>> The API versioning is a good idea but needs to be more holistic across
>>>> the platform I think.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <ad...@gmail.com> wrote:
>>>>
>>>> Hi Kapil,
>>>>
>>>> Thank you for raising the concerns below. I’ll need some additional
>>>> details to fully understand your points:
>>>>
>>>> 1.      *Collection Sheet API* – You mentioned it appears
>>>> non-functional and contains several logical errors.
>>>>
>>>> o    If it’s indeed not working, that’s a separate, high-priority
>>>> discussion.
>>>>
>>>> o    Could you clarify which logical errors you were referring to, and
>>>> what specifically makes you think it’s non-functional?
>>>>
>>>> 2.      *Service annotations* – You noted that service methods are not
>>>> annotated with @Service and that beans are defined manually.
>>>>
>>>> o    Are you referring to the
>>>> CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being
>>>> defined via configuration?
>>>>
>>>> 3.      *Repository wrappers annotated with **@Service* – You
>>>> mentioned that this mandates full unit test coverage but that they should
>>>> ideally be annotated with @Component.
>>>>
>>>> o    Could you point out the exact classes you had in mind?
>>>>
>>>> As for the other points, I agree we can refactor and remove redundant
>>>> logic—please feel free to suggest specific improvements or start work on
>>>> them immediately!
>>>>
>>>> However, be careful by moving anything into the fineract-core… We are
>>>> aiming to keep it as small as possible as everything is built on top of
>>>> this module! If collection sheet are used for loans and savings - for
>>>> example - than the recommended move is NOT to move this logic into core!
>>>>
>>>> Either:
>>>>
>>>> - we split the logic into fineract-loan and fineract-savings
>>>>
>>>> - Move the logic into a new module
>>>>
>>>> - Leave it in fineract-provider for now
>>>>
>>>>
>>>>
>>>> Shall you have any questions, please let us know!
>>>>
>>>> Regards,
>>>> Adam
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On 2025. Aug 11., at 12:09, Kapil Panchal <
>>>> kapil.panchal.development@gmail.com> wrote:
>>>>
>>>>
>>>>
>>>> Hi Adam,
>>>>
>>>> I’m currently working on *FINERACT-2290* and have a few questions
>>>> before I submit a pull request.
>>>>
>>>> The *Collection Sheet API* in its current state appears non-functional
>>>> and contains several logical errors. It seems there was an earlier attempt
>>>> to convert from a JSON string request parameter to a class-based request
>>>> object, but:
>>>>
>>>> Certain fields are missing.
>>>>
>>>> The serializer is not correctly populating the objects, which causes
>>>> the conditional checks to be bypassed and results in incorrect (false)
>>>> responses.
>>>>
>>>> This change set is *high risk* because it touches most of the loan and
>>>> savings product logic. I’ve had to refactor almost all major methods.
>>>> Extensive integration and end-to-end testing will be required to ensure
>>>> there are no regressions, especially in edge cases. At present, there are
>>>> no unit or integration tests for this functionality, and test creation is
>>>> outside the current ticket scope. I’ve been iterating on this for a while,
>>>> and only today have I reached a stable state after several experimental and
>>>> build-breaking attempts.
>>>>
>>>> *Key Observations:*
>>>>
>>>> Service methods are not annotated with @Service; instead, beans are
>>>> defined manually.
>>>>
>>>> Repository wrappers are annotated with @Service. This mandates full
>>>> unit test coverage for these methods, but they should ideally be annotated
>>>> with @Component.
>>>>
>>>> I agree with prior discussions on separating bean validation — having a
>>>> dedicated @Component validation class allows the request object to handle
>>>> checks independent of database queries.
>>>>
>>>> Validation components can also perform database-related validations;
>>>> these can be injected into service classes for cleaner architecture.
>>>>
>>>> Such validation components should be placed in *Fineract-Core* so they
>>>> are reusable across modules, reducing future refactoring needs.
>>>>
>>>> The current design of having commands in *Fineract-Core* and
>>>> handlers/services/repositories in respective modules is good — it cleanly
>>>> decouples command definition from execution.
>>>>
>>>> There is extensive use of this. in singleton contexts (API, Service,
>>>> Repository). While not harmful, it’s unnecessary boilerplate.
>>>>
>>>> Multiple redundant intermediate DTOs exist where the request DTO itself
>>>> could be reused for data transfer.
>>>>
>>>> I found redundant logic — e.g., a for loop with a break statement that
>>>> effectively executes only once; this can be simplified.
>>>>
>>>> Some JDBC template queries use reserved SQL keywords, causing
>>>> exceptions. Refactoring these queries resolves the issue and returns proper
>>>> response objects.
>>>>
>>>> *Suggestions:*
>>>>
>>>> *Where appropriate, large tickets should be broken into subtasks to
>>>> manage complexity and reviewability.*
>>>>
>>>> It may help to have a dedicated *developer-only Slack channel *for
>>>> technical discussions. This could complement other community spaces if
>>>> there’s a need to keep certain conversations more focused.
>>>>
>>>> What are your thoughts on the above?
>>>>
>>>> Thanks,
>>>> Kapil
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> *Ed Cable*
>>>>
>>>> President/CEO, Mifos Initiative
>>>>
>>>> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>>>>
>>>>
>>>>
>>>> *Collectively Creating a World of 3 Billion Maries | *http://mifos.org *
>>>> <http://facebook.com/mifos>  <http://www.twitter.com/mifos>*
>>>>
>>>>
>>>>
>>>>

Re: Collection Sheet Deprecation was [Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)]

[Kapil Panchal <ka...@gmail.com>]

Victor, All,

I can maintain the Collection Sheet efforts for the Project. I would need
all the necessary clearances and permissions for the same. As regards the
functional and non-functional requirements of the project, a separate tool
is needed eg. IBM Rational Doors or similar. The requirements also need to
be revisioned similar to the product revisions/versions that we have. Any
change in the project has to be communicated so that impact analyses can be
done in terms of code changes, code coverage and also the documentation
changes. All IP generated will be documented as a reference.

I would also need a peer reviewer assigned who can review the pull
requests? I will keep the stakeholders posted.

Thanks,
Kapil

On Tue, Sep 2, 2025 at 9:46 AM VICTOR MANUEL ROMERO RODRIGUEZ <
victor.romero@fintecheando.mx> wrote:

> Kapil/Felix,
>
> Is it possible to show/write a description from a user/functional
> perspective?
>
> Kapil
>
> Your description is useful but for the non functional
> requirement (refactoring).
>
> Regards
>
> El lun, 1 sept 2025 a las 21:53, Kapil Panchal (<
> kapil.panchal.development@gmail.com>) escribió:
>
>> The current Collection Sheet efforts started as an attempt to raise a
>> pull request to the Jira Ticket
>> https://issues.apache.org/jira/browse/FINERACT-2290, this ticket aims at
>> solving the technical debt
>> https://issues.apache.org/jira/browse/FINERACT-2169, In other words, it
>> replaces the existing (old/archaic) logic processing (CQRS, Serialization
>> etc) with the new and improved/latest solution which removes much of the
>> boilerplate and aims at making the code lean such that all the testing and
>> related infrastructure needed to manage the quality of the code and
>> deliverables are minimum, it does that by leveraging the framework and the
>> supported libraries itself. The framework is fully tested and baselined, if
>> the code that we write falls inline with framework coding/logic paradigm
>> then most of the testing need not be done as it has already been tested,
>> this is a lot akin to testing the database with ACID. None of our code
>> tests the database/framework/webserver/libraries as they are fully tested
>> and these are the first principles of a good software design, minimal code
>> maximal output.
>>
>> With the above in mind and the instructions in the card itself which
>> assumes that the command processing logic is to be changed and not the
>> business logic itself.
>>
>> Cutting the story short, I have attached herewith a UML Class diagram to
>> elicit why the existing CollectionSheet is non functional. The API accepts
>> 2 parameters as stated below,
>>
>> 1) GenerateCollectionSheet,
>> 2) SaveCollectionSheet.
>>
>> Note1: The first parameter 1) Generate Collection Sheet makes a call to
>> the Service which inturn has a JDBC logic where mysql reserved keywords are
>> used, this always returns an exception. (This could have been a recent
>> mysql release and previously this query might have been working? I am using
>> the mysql version as stated in the .md documentation).
>>
>> Note2: The second parameter 2) A CQRS command that is audited, has three
>> private methods a) updateBulkRepayments, b) updateBulkDisbursals, c)
>> updateBulkMandatorySavingsDuePayments have errors in extracting the json
>> payload one of the methods is mentioned in the UML. There are other errors
>> too that make this Collection Sheet take code paths which do not reflect
>> the true metric.
>>
>> [image: Model_CollectionSheet_v1_Class_Diagram.PNG]
>> An excerpt from the code: The bold code is where the extraction of JSON
>> objects if not done correctly will bypass most of the logic and return
>> flawed response objects. Here it is the case. Please refer to the chain
>> emails where the correct logic to extract a variable from the json payload
>> is highlighted.
>>
>> public Collection<SavingsAccountTransactionDTO>
>> assembleBulkMandatorySavingsAccountTransactionDTOs(final JsonCommand
>> command,
>>
>>         final PaymentDetail paymentDetail) {
>>     final String json = command.json();
>>     if (StringUtils.isBlank(json)) {
>>         throw new InvalidJsonException();
>>     }
>>     final JsonElement element = this.fromApiJsonHelper.parse(json);
>>     final Collection<SavingsAccountTransactionDTO>
>> savingsAccountTransactions = new ArrayList<>();
>>     final LocalDate transactionDate =
>> this.fromApiJsonHelper.extractLocalDateNamed(transactionDateParamName,
>> element);
>>     final String dateFormat =
>> this.fromApiJsonHelper.extractDateFormatParameter(element.getAsJsonObject());
>>     final JsonObject topLevelJsonElement = element.getAsJsonObject();
>>     final Locale locale =
>> this.fromApiJsonHelper.extractLocaleParameter(topLevelJsonElement);
>>     final DateTimeFormatter formatter =
>> DateTimeFormatter.ofPattern(dateFormat).withLocale(locale);
>>
>>
>>
>>
>>
>>
>> * if (element.isJsonObject()) {        if
>> (topLevelJsonElement.has(bulkSavingsDueTransactionsParamName)
>>   &&
>> topLevelJsonElement.get(bulkSavingsDueTransactionsParamName).isJsonArray())
>> {            final JsonArray array =
>> topLevelJsonElement.get(bulkSavingsDueTransactionsParamName).getAsJsonArray();
>>           for (int i = 0; i < array.size(); i++) {*
>>
>>
>> "If you can raise a JIRA for each of the logical errors (and ideally
>> provide add a test case into the automated tests to demonstrate each), then
>> let's first fix these errors. After the API works as expected, we can then
>> proceed to refactor it."
>>
>>
>> I think this effort can be broken down to 3-4 Jira tickets for it to be
>> fully functional. Assuming that we will be using the same command
>> processing logic as we have been using and not the latest one as per
>> https://issues.apache.org/jira/browse/FINERACT-2169.
>>
>> 1) Jira Ticket 1 - Work on getting the Generate Collection Sheet up and
>> running, including unit tests and integration tests.
>> 2) Jira Ticket 2 - Work on getting the Save Collection Sheet up and
>> running, excluding unit tests and/or integration tests (some refactoring).
>> 3) Jira Ticket 3 - Work on getting the Save Collection Sheet unit tests.
>> 4) Jira Ticket 4 - Work on getting the Save Collection Sheet integration
>> tests.
>>
>> The release to production of tickets 2 - 4 can be done when all are
>> approved? Please let me know so that I can start working on those ASAP.
>>
>> Thanks,
>> Kapil
>>
>> On Tue, Sep 2, 2025 at 4:48 AM Petri Tuomola <pe...@gmail.com>
>> wrote:
>>
>>> Thanks Kapil - but this JIRA / PRs seem to be about refactoring the
>>> Collection Sheet API to the new command processing pattern, not about
>>> fixing defects in the API.
>>>
>>> Before we start any refactoring, we should make sure we're starting from
>>> a known good state and that there's sufficient unit testing coverage to
>>> test the functionality, so we know if we've broken something during
>>> refactoring.
>>>
>>> To get to this point, can you share what your comment "The *Collection
>>> Sheet API* in its current state appears non-functional and contains
>>> several logical errors" refers to?
>>>
>>> If you can raise a JIRA for each of the logical errors (and ideally
>>> provide add a test case into the automated tests to demonstrate each), then
>>> let's first fix these errors. After the API works as expected, we can then
>>> proceed to refactor it.
>>>
>>> Regards
>>> Petri
>>>
>>>
>>>
>>> ------------------------------
>>> *From:* Kapil Panchal <ka...@gmail.com>
>>> *Sent:* Monday, September 1, 2025 7:42 PM
>>> *To:* dev@fineract.apache.org <de...@fineract.apache.org>
>>> *Cc:* Edward Cable <ed...@mifos.org>
>>> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
>>> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
>>>
>>> Hi Petri,
>>>
>>> Please take a look at the Jira Ticket
>>> https://issues.apache.org/jira/browse/FINERACT-2290. I had released a
>>> Pull Request https://github.com/apache/fineract/pull/4943.
>>>
>>> Thanks,
>>> Kapil
>>>
>>> On Mon, Sep 1, 2025 at 7:27 AM Petri Tuomola <pe...@gmail.com>
>>> wrote:
>>>
>>> Hi
>>>
>>> I'm happy to help fix issues with Collection Sheet API.
>>>
>>> If I look at the emails that started all this, I can only see this
>>> comment:
>>>
>>> "The *Collection Sheet API* in its current state appears non-functional
>>> and contains several logical errors"
>>>
>>>
>>> I think it's fair to expect a bit more detailed bug report than that.
>>>
>>> Is there any more details on this issue: how is it non-functional, and
>>> what are the logical errors?
>>>
>>>
>>> Who can provide the list of JIRAs that need to be fixed? I can then have
>>> a look this weekend.
>>>
>>>
>>> Regards
>>>
>>> Petri
>>>
>>>
>>>
>>> ------------------------------
>>> *From:* James Dailey <jd...@apache.org>
>>> *Sent:* Saturday, August 30, 2025 3:01 AM
>>> *To:* dev@fineract.apache.org <de...@fineract.apache.org>; Edward Cable <
>>> edcable@mifos.org>
>>> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
>>> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
>>>
>>> Devs -  Are we done with this?
>>>
>>> First, I am NOT ruling out this functionality as important but if no one
>>> is actually demanding it, and no one is willing to maintain it, then it
>>> doesn't get maintained.  That is the nature of open source projects.
>>>
>>> Second, it can happen that an outside vendor (e.g. Mifos) that relies
>>> on this Collection Sheet functionality (or any functionality) inadvertently
>>> drops that from their front end release.  Because the tests at Fineract do
>>> not cover this fully, no one at the Vendor and no one at the Fineract
>>> project will see the functionality fail in a build until a customer or
>>> implementer notices. So, tests are vital for ensuring ongoing
>>> maintainability.
>>>
>>> Third, if this type of thing isn't "naturally" part of the core - then
>>> it will make a lot more sense to have it be an outside extension - in which
>>> case the tests have to be written for the API calls, and a Vendor should
>>> try to get such tests contributed.   I could be wrong about this, I would
>>> be happy to debate where this belongs.
>>>
>>> @Edward Cable <ed...@mifos.org> - what is the assessment and status
>>> of needed functionality and where do you think it should live?
>>>
>>> Thanks,
>>> James
>>>
>>> On Mon, Aug 18, 2025 at 8:21 AM James Dailey <jd...@apache.org> wrote:
>>>
>>> Sifiso -
>>>
>>> Do we have a way to reach the tens of thousands of institutions you
>>> believe are there?
>>>
>>> My belief (just anecdotal information) is that it’s more like a few
>>> hundred institutions and that out of that number, less than 10% are on a
>>> recent release.  And, of that smaller number, few, if any, are still using
>>> Collection Sheets.  I could be wrong - I’d like the data!  I’d say that
>>> vendors can show up with their data here to help make this vendor neutral
>>> space more informed.
>>>
>>> If group lending and group-member lending and collection sheets are
>>> still needed, then perhaps those project volunteers can contribute the
>>> needed updates to keep the functionality useful.
>>>
>>> Thanks
>>> Thanks
>>>
>>>
>>> On Mon, Aug 18, 2025 at 6:48 AM Sifiso Mtetwa <si...@skyburgsystems.org>
>>> wrote:
>>>
>>> Hi guys,
>>>
>>>
>>>
>>> This is an interesting topic. I have been wondering why, in general, we
>>> seem to be deprecating more and more system functions. The individual
>>> collection sheet has served us well over the years and still does, If
>>> anything we could improve on its functionality by maybe adding a bulk
>>> collection sheet template with little detail compared to a full loan
>>> repayment bulk import template.
>>>
>>>
>>>
>>> Fineract is used by tens of thousands of organisations throughout the
>>> world and most of them are not on this listing and may not have a voice to
>>> air their concerns. Maybe we can find a way of exposing this thread to
>>> include more voters.
>>>
>>>
>>>
>>> Regards,
>>>
>>>
>>>
>>> *From:* James Dailey [mailto:jdailey@apache.org]
>>> *Sent:* Saturday, 16 August 2025 02:53
>>> *To:* dev@fineract.apache.org
>>> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
>>> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
>>>
>>>
>>>
>>> Ed
>>>
>>>
>>>
>>> I agree this needs to be discussed but it is important to acknowledge
>>> that THIS Fineract listserv is the only official (and required) discussion
>>> space.  It is not the intention to bury anything in “an email thread”.
>>>
>>>
>>>
>>> When I started up Mifos we spent a lot of time looking at Collection
>>> Sheets and designing process flows around them. I fully know that this
>>> design direction was important back then in 2002-2006.
>>>
>>>
>>>
>>> However if there is no one here asking for them to be retained besides
>>> you, then that is a sign that they have perhaps reached an end of their
>>> utility.  Or, that the users are not actually here, which is a different
>>> problem.
>>>
>>>
>>>
>>> So,… If there is a group using collection sheets in production AND they
>>> are not on some permanent forked (old) version, then now is the time to
>>> speak up.  Here.
>>>
>>>
>>>
>>> Generally, I’m fairly certain we can refactor this with an eye toward
>>> extracting it from the core.  Repeating the logic in two places makes no
>>> sense either.  Collection sheets are kind of assembled from
>>> constituent loans and savings.  The balances and due payments should be
>>> calculated in the underlying components.
>>>
>>>
>>>
>>> As the system gets restructured we need to decide to keep this at all,
>>> to keep it in a new place, or as some external concept/plug in. Why
>>> wouldn’t we want a separate component?
>>>
>>>
>>>
>>> Cheers
>>>
>>>
>>>
>>> On Thu, Aug 14, 2025 at 4:16 PM Ed Cable <ed...@mifos.org> wrote:
>>>
>>> I'll leave the other thread for discussion of the API versioning and
>>> refactoring related to Collection Sheet API but wanted to create a separate
>>> thread regarding the deprecation of the Collection Sheet.
>>>
>>>
>>>
>>> In general, for this and removal of any functionality, it's something
>>> that needs to be discussed openly with the community and with a vote and
>>> not a decision buried in a mailing list thread.
>>>
>>>
>>>
>>> For the collection sheet specifically, more thought has to be given to
>>> its deprecation as the centrality and highly coupled nature of the
>>> collection sheet is being understated as it isn't merely a report that's to
>>> be printed or a form filled out via a mobile application. It's a
>>> significant portion of the user interface and highly coupled to many of the
>>> microfinance features around groups/centers/meeting scheduling, staff
>>> assignment, etc.
>>>
>>>
>>>
>>> I do agree that from a UI perspective, the collection sheet and other
>>> microfinance-centric functionalities and flows should be viewable based on
>>> a configurable setting. As it doesn't lend itself to the optimal user
>>> experience for a large portion of current Fineract user base.
>>>
>>>
>>>
>>> I also am supportive of a strategy of slimming Fineract to its core
>>> services and functionality above core Fineract services and APIs can be
>>> extracted out.
>>>
>>>
>>>
>>> So I do think we should give thoughtful consideration to what
>>> abstracting out the collection sheet and corresponding microfinance
>>> functionality would look like and what that effort would entail to abstract
>>> it out without adversely impacting the original user base of the software
>>> but it's not as simple as deprecating these API.
>>>
>>>
>>>
>>> I welcome others' thoughts and inputs as I know even with microfinance
>>> itself, the methodology has evolved and group lending and the concept of a
>>> collection sheet isn't as central as it once was
>>>
>>>
>>>
>>> Thanks,
>>>
>>>
>>>
>>> Ed
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Wed, Aug 13, 2025 at 6:36 PM Kapil Panchal <
>>> kapil.panchal.development@gmail.com> wrote:
>>>
>>> Hi James,
>>>
>>>
>>>
>>> I can proceed by marking this feature as @Deprecated and/or performing
>>> a safe refactor to remove the API.
>>>
>>>
>>>
>>> For API versioning I agree to Aleksandar Vidakovic's proposal on
>>> adapting to the SpringBoot v7/Spring Framework v4. If I may, Aleksandar
>>> Vidakovic takes the lead on this project and I can help to support the
>>> conversion?
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Kapil
>>>
>>>
>>>
>>> On Thu, Aug 14, 2025 at 5:07 AM James Dailey <jd...@apache.org> wrote:
>>>
>>> Hi Kapil
>>>
>>>
>>>
>>> I might suggest looking at this as an opportunity to remove the
>>> collection sheet entirely from the Fineract namespace.  It’s a legacy
>>> concept I and others designed a long time ago, originally in 2002 based on
>>> collection sheets we gathered from a dozen countries. It is strongly tied
>>> to concepts in microfinance field operations, and especially when there was
>>> no data connectivity.
>>>
>>>
>>>
>>> It belongs perhaps as a sort of external microservice - data loading via
>>> a bulk import could still be enabled.
>>>
>>> The API versioning is a good idea but needs to be more holistic across
>>> the platform I think.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <ad...@gmail.com> wrote:
>>>
>>> Hi Kapil,
>>>
>>> Thank you for raising the concerns below. I’ll need some additional
>>> details to fully understand your points:
>>>
>>> 1.      *Collection Sheet API* – You mentioned it appears
>>> non-functional and contains several logical errors.
>>>
>>> o    If it’s indeed not working, that’s a separate, high-priority
>>> discussion.
>>>
>>> o    Could you clarify which logical errors you were referring to, and
>>> what specifically makes you think it’s non-functional?
>>>
>>> 2.      *Service annotations* – You noted that service methods are not
>>> annotated with @Service and that beans are defined manually.
>>>
>>> o    Are you referring to the
>>> CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being defined
>>> via configuration?
>>>
>>> 3.      *Repository wrappers annotated with **@Service* – You mentioned
>>> that this mandates full unit test coverage but that they should ideally be
>>> annotated with @Component.
>>>
>>> o    Could you point out the exact classes you had in mind?
>>>
>>> As for the other points, I agree we can refactor and remove redundant
>>> logic—please feel free to suggest specific improvements or start work on
>>> them immediately!
>>>
>>> However, be careful by moving anything into the fineract-core… We are
>>> aiming to keep it as small as possible as everything is built on top of
>>> this module! If collection sheet are used for loans and savings - for
>>> example - than the recommended move is NOT to move this logic into core!
>>>
>>> Either:
>>>
>>> - we split the logic into fineract-loan and fineract-savings
>>>
>>> - Move the logic into a new module
>>>
>>> - Leave it in fineract-provider for now
>>>
>>>
>>>
>>> Shall you have any questions, please let us know!
>>>
>>> Regards,
>>> Adam
>>>
>>>
>>>
>>>
>>>
>>> On 2025. Aug 11., at 12:09, Kapil Panchal <
>>> kapil.panchal.development@gmail.com> wrote:
>>>
>>>
>>>
>>> Hi Adam,
>>>
>>> I’m currently working on *FINERACT-2290* and have a few questions
>>> before I submit a pull request.
>>>
>>> The *Collection Sheet API* in its current state appears non-functional
>>> and contains several logical errors. It seems there was an earlier attempt
>>> to convert from a JSON string request parameter to a class-based request
>>> object, but:
>>>
>>> Certain fields are missing.
>>>
>>> The serializer is not correctly populating the objects, which causes the
>>> conditional checks to be bypassed and results in incorrect (false)
>>> responses.
>>>
>>> This change set is *high risk* because it touches most of the loan and
>>> savings product logic. I’ve had to refactor almost all major methods.
>>> Extensive integration and end-to-end testing will be required to ensure
>>> there are no regressions, especially in edge cases. At present, there are
>>> no unit or integration tests for this functionality, and test creation is
>>> outside the current ticket scope. I’ve been iterating on this for a while,
>>> and only today have I reached a stable state after several experimental and
>>> build-breaking attempts.
>>>
>>> *Key Observations:*
>>>
>>> Service methods are not annotated with @Service; instead, beans are
>>> defined manually.
>>>
>>> Repository wrappers are annotated with @Service. This mandates full unit
>>> test coverage for these methods, but they should ideally be annotated with
>>> @Component.
>>>
>>> I agree with prior discussions on separating bean validation — having a
>>> dedicated @Component validation class allows the request object to handle
>>> checks independent of database queries.
>>>
>>> Validation components can also perform database-related validations;
>>> these can be injected into service classes for cleaner architecture.
>>>
>>> Such validation components should be placed in *Fineract-Core* so they
>>> are reusable across modules, reducing future refactoring needs.
>>>
>>> The current design of having commands in *Fineract-Core* and
>>> handlers/services/repositories in respective modules is good — it cleanly
>>> decouples command definition from execution.
>>>
>>> There is extensive use of this. in singleton contexts (API, Service,
>>> Repository). While not harmful, it’s unnecessary boilerplate.
>>>
>>> Multiple redundant intermediate DTOs exist where the request DTO itself
>>> could be reused for data transfer.
>>>
>>> I found redundant logic — e.g., a for loop with a break statement that
>>> effectively executes only once; this can be simplified.
>>>
>>> Some JDBC template queries use reserved SQL keywords, causing
>>> exceptions. Refactoring these queries resolves the issue and returns proper
>>> response objects.
>>>
>>> *Suggestions:*
>>>
>>> *Where appropriate, large tickets should be broken into subtasks to
>>> manage complexity and reviewability.*
>>>
>>> It may help to have a dedicated *developer-only Slack channel *for
>>> technical discussions. This could complement other community spaces if
>>> there’s a need to keep certain conversations more focused.
>>>
>>> What are your thoughts on the above?
>>>
>>> Thanks,
>>> Kapil
>>>
>>>
>>>
>>>
>>>
>>>
>>> --
>>>
>>> *Ed Cable*
>>>
>>> President/CEO, Mifos Initiative
>>>
>>> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>>>
>>>
>>>
>>> *Collectively Creating a World of 3 Billion Maries | *http://mifos.org *
>>> <http://facebook.com/mifos>  <http://www.twitter.com/mifos>*
>>>
>>>
>>>
>>>

Re: Collection Sheet Deprecation was [Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)]

[VICTOR MANUEL ROMERO RODRIGUEZ <vi...@fintecheando.mx>]

Kapil/Felix,

Is it possible to show/write a description from a user/functional
perspective?

Kapil

Your description is useful but for the non functional
requirement (refactoring).

Regards

El lun, 1 sept 2025 a las 21:53, Kapil Panchal (<
kapil.panchal.development@gmail.com>) escribió:

> The current Collection Sheet efforts started as an attempt to raise a pull
> request to the Jira Ticket
> https://issues.apache.org/jira/browse/FINERACT-2290, this ticket aims at
> solving the technical debt
> https://issues.apache.org/jira/browse/FINERACT-2169, In other words, it
> replaces the existing (old/archaic) logic processing (CQRS, Serialization
> etc) with the new and improved/latest solution which removes much of the
> boilerplate and aims at making the code lean such that all the testing and
> related infrastructure needed to manage the quality of the code and
> deliverables are minimum, it does that by leveraging the framework and the
> supported libraries itself. The framework is fully tested and baselined, if
> the code that we write falls inline with framework coding/logic paradigm
> then most of the testing need not be done as it has already been tested,
> this is a lot akin to testing the database with ACID. None of our code
> tests the database/framework/webserver/libraries as they are fully tested
> and these are the first principles of a good software design, minimal code
> maximal output.
>
> With the above in mind and the instructions in the card itself which
> assumes that the command processing logic is to be changed and not the
> business logic itself.
>
> Cutting the story short, I have attached herewith a UML Class diagram to
> elicit why the existing CollectionSheet is non functional. The API accepts
> 2 parameters as stated below,
>
> 1) GenerateCollectionSheet,
> 2) SaveCollectionSheet.
>
> Note1: The first parameter 1) Generate Collection Sheet makes a call to
> the Service which inturn has a JDBC logic where mysql reserved keywords are
> used, this always returns an exception. (This could have been a recent
> mysql release and previously this query might have been working? I am using
> the mysql version as stated in the .md documentation).
>
> Note2: The second parameter 2) A CQRS command that is audited, has three
> private methods a) updateBulkRepayments, b) updateBulkDisbursals, c)
> updateBulkMandatorySavingsDuePayments have errors in extracting the json
> payload one of the methods is mentioned in the UML. There are other errors
> too that make this Collection Sheet take code paths which do not reflect
> the true metric.
>
> [image: Model_CollectionSheet_v1_Class_Diagram.PNG]
> An excerpt from the code: The bold code is where the extraction of JSON
> objects if not done correctly will bypass most of the logic and return
> flawed response objects. Here it is the case. Please refer to the chain
> emails where the correct logic to extract a variable from the json payload
> is highlighted.
>
> public Collection<SavingsAccountTransactionDTO>
> assembleBulkMandatorySavingsAccountTransactionDTOs(final JsonCommand
> command,
>
>         final PaymentDetail paymentDetail) {
>     final String json = command.json();
>     if (StringUtils.isBlank(json)) {
>         throw new InvalidJsonException();
>     }
>     final JsonElement element = this.fromApiJsonHelper.parse(json);
>     final Collection<SavingsAccountTransactionDTO>
> savingsAccountTransactions = new ArrayList<>();
>     final LocalDate transactionDate =
> this.fromApiJsonHelper.extractLocalDateNamed(transactionDateParamName,
> element);
>     final String dateFormat =
> this.fromApiJsonHelper.extractDateFormatParameter(element.getAsJsonObject());
>     final JsonObject topLevelJsonElement = element.getAsJsonObject();
>     final Locale locale =
> this.fromApiJsonHelper.extractLocaleParameter(topLevelJsonElement);
>     final DateTimeFormatter formatter =
> DateTimeFormatter.ofPattern(dateFormat).withLocale(locale);
>
>
>
>
>
>
> * if (element.isJsonObject()) {        if
> (topLevelJsonElement.has(bulkSavingsDueTransactionsParamName)
>   &&
> topLevelJsonElement.get(bulkSavingsDueTransactionsParamName).isJsonArray())
> {            final JsonArray array =
> topLevelJsonElement.get(bulkSavingsDueTransactionsParamName).getAsJsonArray();
>           for (int i = 0; i < array.size(); i++) {*
>
>
> "If you can raise a JIRA for each of the logical errors (and ideally
> provide add a test case into the automated tests to demonstrate each), then
> let's first fix these errors. After the API works as expected, we can then
> proceed to refactor it."
>
>
> I think this effort can be broken down to 3-4 Jira tickets for it to be
> fully functional. Assuming that we will be using the same command
> processing logic as we have been using and not the latest one as per
> https://issues.apache.org/jira/browse/FINERACT-2169.
>
> 1) Jira Ticket 1 - Work on getting the Generate Collection Sheet up and
> running, including unit tests and integration tests.
> 2) Jira Ticket 2 - Work on getting the Save Collection Sheet up and
> running, excluding unit tests and/or integration tests (some refactoring).
> 3) Jira Ticket 3 - Work on getting the Save Collection Sheet unit tests.
> 4) Jira Ticket 4 - Work on getting the Save Collection Sheet integration
> tests.
>
> The release to production of tickets 2 - 4 can be done when all are
> approved? Please let me know so that I can start working on those ASAP.
>
> Thanks,
> Kapil
>
> On Tue, Sep 2, 2025 at 4:48 AM Petri Tuomola <pe...@gmail.com>
> wrote:
>
>> Thanks Kapil - but this JIRA / PRs seem to be about refactoring the
>> Collection Sheet API to the new command processing pattern, not about
>> fixing defects in the API.
>>
>> Before we start any refactoring, we should make sure we're starting from
>> a known good state and that there's sufficient unit testing coverage to
>> test the functionality, so we know if we've broken something during
>> refactoring.
>>
>> To get to this point, can you share what your comment "The *Collection
>> Sheet API* in its current state appears non-functional and contains
>> several logical errors" refers to?
>>
>> If you can raise a JIRA for each of the logical errors (and ideally
>> provide add a test case into the automated tests to demonstrate each), then
>> let's first fix these errors. After the API works as expected, we can then
>> proceed to refactor it.
>>
>> Regards
>> Petri
>>
>>
>>
>> ------------------------------
>> *From:* Kapil Panchal <ka...@gmail.com>
>> *Sent:* Monday, September 1, 2025 7:42 PM
>> *To:* dev@fineract.apache.org <de...@fineract.apache.org>
>> *Cc:* Edward Cable <ed...@mifos.org>
>> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
>> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
>>
>> Hi Petri,
>>
>> Please take a look at the Jira Ticket
>> https://issues.apache.org/jira/browse/FINERACT-2290. I had released a
>> Pull Request https://github.com/apache/fineract/pull/4943.
>>
>> Thanks,
>> Kapil
>>
>> On Mon, Sep 1, 2025 at 7:27 AM Petri Tuomola <pe...@gmail.com>
>> wrote:
>>
>> Hi
>>
>> I'm happy to help fix issues with Collection Sheet API.
>>
>> If I look at the emails that started all this, I can only see this
>> comment:
>>
>> "The *Collection Sheet API* in its current state appears non-functional
>> and contains several logical errors"
>>
>>
>> I think it's fair to expect a bit more detailed bug report than that.
>>
>> Is there any more details on this issue: how is it non-functional, and
>> what are the logical errors?
>>
>>
>> Who can provide the list of JIRAs that need to be fixed? I can then have
>> a look this weekend.
>>
>>
>> Regards
>>
>> Petri
>>
>>
>>
>> ------------------------------
>> *From:* James Dailey <jd...@apache.org>
>> *Sent:* Saturday, August 30, 2025 3:01 AM
>> *To:* dev@fineract.apache.org <de...@fineract.apache.org>; Edward Cable <
>> edcable@mifos.org>
>> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
>> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
>>
>> Devs -  Are we done with this?
>>
>> First, I am NOT ruling out this functionality as important but if no one
>> is actually demanding it, and no one is willing to maintain it, then it
>> doesn't get maintained.  That is the nature of open source projects.
>>
>> Second, it can happen that an outside vendor (e.g. Mifos) that relies
>> on this Collection Sheet functionality (or any functionality) inadvertently
>> drops that from their front end release.  Because the tests at Fineract do
>> not cover this fully, no one at the Vendor and no one at the Fineract
>> project will see the functionality fail in a build until a customer or
>> implementer notices. So, tests are vital for ensuring ongoing
>> maintainability.
>>
>> Third, if this type of thing isn't "naturally" part of the core - then it
>> will make a lot more sense to have it be an outside extension - in which
>> case the tests have to be written for the API calls, and a Vendor should
>> try to get such tests contributed.   I could be wrong about this, I would
>> be happy to debate where this belongs.
>>
>> @Edward Cable <ed...@mifos.org> - what is the assessment and status of
>> needed functionality and where do you think it should live?
>>
>> Thanks,
>> James
>>
>> On Mon, Aug 18, 2025 at 8:21 AM James Dailey <jd...@apache.org> wrote:
>>
>> Sifiso -
>>
>> Do we have a way to reach the tens of thousands of institutions you
>> believe are there?
>>
>> My belief (just anecdotal information) is that it’s more like a few
>> hundred institutions and that out of that number, less than 10% are on a
>> recent release.  And, of that smaller number, few, if any, are still using
>> Collection Sheets.  I could be wrong - I’d like the data!  I’d say that
>> vendors can show up with their data here to help make this vendor neutral
>> space more informed.
>>
>> If group lending and group-member lending and collection sheets are still
>> needed, then perhaps those project volunteers can contribute the needed
>> updates to keep the functionality useful.
>>
>> Thanks
>> Thanks
>>
>>
>> On Mon, Aug 18, 2025 at 6:48 AM Sifiso Mtetwa <si...@skyburgsystems.org>
>> wrote:
>>
>> Hi guys,
>>
>>
>>
>> This is an interesting topic. I have been wondering why, in general, we
>> seem to be deprecating more and more system functions. The individual
>> collection sheet has served us well over the years and still does, If
>> anything we could improve on its functionality by maybe adding a bulk
>> collection sheet template with little detail compared to a full loan
>> repayment bulk import template.
>>
>>
>>
>> Fineract is used by tens of thousands of organisations throughout the
>> world and most of them are not on this listing and may not have a voice to
>> air their concerns. Maybe we can find a way of exposing this thread to
>> include more voters.
>>
>>
>>
>> Regards,
>>
>>
>>
>> *From:* James Dailey [mailto:jdailey@apache.org]
>> *Sent:* Saturday, 16 August 2025 02:53
>> *To:* dev@fineract.apache.org
>> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
>> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
>>
>>
>>
>> Ed
>>
>>
>>
>> I agree this needs to be discussed but it is important to acknowledge
>> that THIS Fineract listserv is the only official (and required) discussion
>> space.  It is not the intention to bury anything in “an email thread”.
>>
>>
>>
>> When I started up Mifos we spent a lot of time looking at Collection
>> Sheets and designing process flows around them. I fully know that this
>> design direction was important back then in 2002-2006.
>>
>>
>>
>> However if there is no one here asking for them to be retained besides
>> you, then that is a sign that they have perhaps reached an end of their
>> utility.  Or, that the users are not actually here, which is a different
>> problem.
>>
>>
>>
>> So,… If there is a group using collection sheets in production AND they
>> are not on some permanent forked (old) version, then now is the time to
>> speak up.  Here.
>>
>>
>>
>> Generally, I’m fairly certain we can refactor this with an eye toward
>> extracting it from the core.  Repeating the logic in two places makes no
>> sense either.  Collection sheets are kind of assembled from
>> constituent loans and savings.  The balances and due payments should be
>> calculated in the underlying components.
>>
>>
>>
>> As the system gets restructured we need to decide to keep this at all, to
>> keep it in a new place, or as some external concept/plug in. Why wouldn’t
>> we want a separate component?
>>
>>
>>
>> Cheers
>>
>>
>>
>> On Thu, Aug 14, 2025 at 4:16 PM Ed Cable <ed...@mifos.org> wrote:
>>
>> I'll leave the other thread for discussion of the API versioning and
>> refactoring related to Collection Sheet API but wanted to create a separate
>> thread regarding the deprecation of the Collection Sheet.
>>
>>
>>
>> In general, for this and removal of any functionality, it's something
>> that needs to be discussed openly with the community and with a vote and
>> not a decision buried in a mailing list thread.
>>
>>
>>
>> For the collection sheet specifically, more thought has to be given to
>> its deprecation as the centrality and highly coupled nature of the
>> collection sheet is being understated as it isn't merely a report that's to
>> be printed or a form filled out via a mobile application. It's a
>> significant portion of the user interface and highly coupled to many of the
>> microfinance features around groups/centers/meeting scheduling, staff
>> assignment, etc.
>>
>>
>>
>> I do agree that from a UI perspective, the collection sheet and other
>> microfinance-centric functionalities and flows should be viewable based on
>> a configurable setting. As it doesn't lend itself to the optimal user
>> experience for a large portion of current Fineract user base.
>>
>>
>>
>> I also am supportive of a strategy of slimming Fineract to its core
>> services and functionality above core Fineract services and APIs can be
>> extracted out.
>>
>>
>>
>> So I do think we should give thoughtful consideration to what abstracting
>> out the collection sheet and corresponding microfinance functionality would
>> look like and what that effort would entail to abstract it out without
>> adversely impacting the original user base of the software but it's not as
>> simple as deprecating these API.
>>
>>
>>
>> I welcome others' thoughts and inputs as I know even with microfinance
>> itself, the methodology has evolved and group lending and the concept of a
>> collection sheet isn't as central as it once was
>>
>>
>>
>> Thanks,
>>
>>
>>
>> Ed
>>
>>
>>
>>
>>
>>
>>
>> On Wed, Aug 13, 2025 at 6:36 PM Kapil Panchal <
>> kapil.panchal.development@gmail.com> wrote:
>>
>> Hi James,
>>
>>
>>
>> I can proceed by marking this feature as @Deprecated and/or performing a
>> safe refactor to remove the API.
>>
>>
>>
>> For API versioning I agree to Aleksandar Vidakovic's proposal on adapting
>> to the SpringBoot v7/Spring Framework v4. If I may, Aleksandar
>> Vidakovic takes the lead on this project and I can help to support the
>> conversion?
>>
>>
>>
>> Thanks,
>>
>> Kapil
>>
>>
>>
>> On Thu, Aug 14, 2025 at 5:07 AM James Dailey <jd...@apache.org> wrote:
>>
>> Hi Kapil
>>
>>
>>
>> I might suggest looking at this as an opportunity to remove the
>> collection sheet entirely from the Fineract namespace.  It’s a legacy
>> concept I and others designed a long time ago, originally in 2002 based on
>> collection sheets we gathered from a dozen countries. It is strongly tied
>> to concepts in microfinance field operations, and especially when there was
>> no data connectivity.
>>
>>
>>
>> It belongs perhaps as a sort of external microservice - data loading via
>> a bulk import could still be enabled.
>>
>> The API versioning is a good idea but needs to be more holistic across
>> the platform I think.
>>
>>
>>
>>
>>
>>
>>
>> On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <ad...@gmail.com> wrote:
>>
>> Hi Kapil,
>>
>> Thank you for raising the concerns below. I’ll need some additional
>> details to fully understand your points:
>>
>> 1.      *Collection Sheet API* – You mentioned it appears non-functional
>> and contains several logical errors.
>>
>> o    If it’s indeed not working, that’s a separate, high-priority
>> discussion.
>>
>> o    Could you clarify which logical errors you were referring to, and
>> what specifically makes you think it’s non-functional?
>>
>> 2.      *Service annotations* – You noted that service methods are not
>> annotated with @Service and that beans are defined manually.
>>
>> o    Are you referring to the
>> CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being defined
>> via configuration?
>>
>> 3.      *Repository wrappers annotated with **@Service* – You mentioned
>> that this mandates full unit test coverage but that they should ideally be
>> annotated with @Component.
>>
>> o    Could you point out the exact classes you had in mind?
>>
>> As for the other points, I agree we can refactor and remove redundant
>> logic—please feel free to suggest specific improvements or start work on
>> them immediately!
>>
>> However, be careful by moving anything into the fineract-core… We are
>> aiming to keep it as small as possible as everything is built on top of
>> this module! If collection sheet are used for loans and savings - for
>> example - than the recommended move is NOT to move this logic into core!
>>
>> Either:
>>
>> - we split the logic into fineract-loan and fineract-savings
>>
>> - Move the logic into a new module
>>
>> - Leave it in fineract-provider for now
>>
>>
>>
>> Shall you have any questions, please let us know!
>>
>> Regards,
>> Adam
>>
>>
>>
>>
>>
>> On 2025. Aug 11., at 12:09, Kapil Panchal <
>> kapil.panchal.development@gmail.com> wrote:
>>
>>
>>
>> Hi Adam,
>>
>> I’m currently working on *FINERACT-2290* and have a few questions before
>> I submit a pull request.
>>
>> The *Collection Sheet API* in its current state appears non-functional
>> and contains several logical errors. It seems there was an earlier attempt
>> to convert from a JSON string request parameter to a class-based request
>> object, but:
>>
>> Certain fields are missing.
>>
>> The serializer is not correctly populating the objects, which causes the
>> conditional checks to be bypassed and results in incorrect (false)
>> responses.
>>
>> This change set is *high risk* because it touches most of the loan and
>> savings product logic. I’ve had to refactor almost all major methods.
>> Extensive integration and end-to-end testing will be required to ensure
>> there are no regressions, especially in edge cases. At present, there are
>> no unit or integration tests for this functionality, and test creation is
>> outside the current ticket scope. I’ve been iterating on this for a while,
>> and only today have I reached a stable state after several experimental and
>> build-breaking attempts.
>>
>> *Key Observations:*
>>
>> Service methods are not annotated with @Service; instead, beans are
>> defined manually.
>>
>> Repository wrappers are annotated with @Service. This mandates full unit
>> test coverage for these methods, but they should ideally be annotated with
>> @Component.
>>
>> I agree with prior discussions on separating bean validation — having a
>> dedicated @Component validation class allows the request object to handle
>> checks independent of database queries.
>>
>> Validation components can also perform database-related validations;
>> these can be injected into service classes for cleaner architecture.
>>
>> Such validation components should be placed in *Fineract-Core* so they
>> are reusable across modules, reducing future refactoring needs.
>>
>> The current design of having commands in *Fineract-Core* and
>> handlers/services/repositories in respective modules is good — it cleanly
>> decouples command definition from execution.
>>
>> There is extensive use of this. in singleton contexts (API, Service,
>> Repository). While not harmful, it’s unnecessary boilerplate.
>>
>> Multiple redundant intermediate DTOs exist where the request DTO itself
>> could be reused for data transfer.
>>
>> I found redundant logic — e.g., a for loop with a break statement that
>> effectively executes only once; this can be simplified.
>>
>> Some JDBC template queries use reserved SQL keywords, causing exceptions.
>> Refactoring these queries resolves the issue and returns proper response
>> objects.
>>
>> *Suggestions:*
>>
>> *Where appropriate, large tickets should be broken into subtasks to
>> manage complexity and reviewability.*
>>
>> It may help to have a dedicated *developer-only Slack channel *for
>> technical discussions. This could complement other community spaces if
>> there’s a need to keep certain conversations more focused.
>>
>> What are your thoughts on the above?
>>
>> Thanks,
>> Kapil
>>
>>
>>
>>
>>
>>
>> --
>>
>> *Ed Cable*
>>
>> President/CEO, Mifos Initiative
>>
>> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>>
>>
>>
>> *Collectively Creating a World of 3 Billion Maries | *http://mifos.org *
>> <http://facebook.com/mifos>  <http://www.twitter.com/mifos>*
>>
>>
>>
>>

Re: Collection Sheet Deprecation was [Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)]

[Kapil Panchal <ka...@gmail.com>]

The current Collection Sheet efforts started as an attempt to raise a pull
request to the Jira Ticket
https://issues.apache.org/jira/browse/FINERACT-2290, this ticket aims at
solving the technical debt
https://issues.apache.org/jira/browse/FINERACT-2169, In other words, it
replaces the existing (old/archaic) logic processing (CQRS, Serialization
etc) with the new and improved/latest solution which removes much of the
boilerplate and aims at making the code lean such that all the testing and
related infrastructure needed to manage the quality of the code and
deliverables are minimum, it does that by leveraging the framework and the
supported libraries itself. The framework is fully tested and baselined, if
the code that we write falls inline with framework coding/logic paradigm
then most of the testing need not be done as it has already been tested,
this is a lot akin to testing the database with ACID. None of our code
tests the database/framework/webserver/libraries as they are fully tested
and these are the first principles of a good software design, minimal code
maximal output.

With the above in mind and the instructions in the card itself which
assumes that the command processing logic is to be changed and not the
business logic itself.

Cutting the story short, I have attached herewith a UML Class diagram to
elicit why the existing CollectionSheet is non functional. The API accepts
2 parameters as stated below,

1) GenerateCollectionSheet,
2) SaveCollectionSheet.

Note1: The first parameter 1) Generate Collection Sheet makes a call to the
Service which inturn has a JDBC logic where mysql reserved keywords are
used, this always returns an exception. (This could have been a recent
mysql release and previously this query might have been working? I am using
the mysql version as stated in the .md documentation).

Note2: The second parameter 2) A CQRS command that is audited, has three
private methods a) updateBulkRepayments, b) updateBulkDisbursals, c)
updateBulkMandatorySavingsDuePayments have errors in extracting the json
payload one of the methods is mentioned in the UML. There are other errors
too that make this Collection Sheet take code paths which do not reflect
the true metric.

[image: Model_CollectionSheet_v1_Class_Diagram.PNG]
An excerpt from the code: The bold code is where the extraction of JSON
objects if not done correctly will bypass most of the logic and return
flawed response objects. Here it is the case. Please refer to the chain
emails where the correct logic to extract a variable from the json payload
is highlighted.

public Collection<SavingsAccountTransactionDTO>
assembleBulkMandatorySavingsAccountTransactionDTOs(final JsonCommand
command,

        final PaymentDetail paymentDetail) {
    final String json = command.json();
    if (StringUtils.isBlank(json)) {
        throw new InvalidJsonException();
    }
    final JsonElement element = this.fromApiJsonHelper.parse(json);
    final Collection<SavingsAccountTransactionDTO>
savingsAccountTransactions = new ArrayList<>();
    final LocalDate transactionDate =
this.fromApiJsonHelper.extractLocalDateNamed(transactionDateParamName,
element);
    final String dateFormat =
this.fromApiJsonHelper.extractDateFormatParameter(element.getAsJsonObject());
    final JsonObject topLevelJsonElement = element.getAsJsonObject();
    final Locale locale =
this.fromApiJsonHelper.extractLocaleParameter(topLevelJsonElement);
    final DateTimeFormatter formatter =
DateTimeFormatter.ofPattern(dateFormat).withLocale(locale);






* if (element.isJsonObject()) {        if
(topLevelJsonElement.has(bulkSavingsDueTransactionsParamName)
  &&
topLevelJsonElement.get(bulkSavingsDueTransactionsParamName).isJsonArray())
{            final JsonArray array =
topLevelJsonElement.get(bulkSavingsDueTransactionsParamName).getAsJsonArray();
          for (int i = 0; i < array.size(); i++) {*


"If you can raise a JIRA for each of the logical errors (and ideally
provide add a test case into the automated tests to demonstrate each), then
let's first fix these errors. After the API works as expected, we can then
proceed to refactor it."


I think this effort can be broken down to 3-4 Jira tickets for it to be
fully functional. Assuming that we will be using the same command
processing logic as we have been using and not the latest one as per
https://issues.apache.org/jira/browse/FINERACT-2169.

1) Jira Ticket 1 - Work on getting the Generate Collection Sheet up and
running, including unit tests and integration tests.
2) Jira Ticket 2 - Work on getting the Save Collection Sheet up and
running, excluding unit tests and/or integration tests (some refactoring).
3) Jira Ticket 3 - Work on getting the Save Collection Sheet unit tests.
4) Jira Ticket 4 - Work on getting the Save Collection Sheet integration
tests.

The release to production of tickets 2 - 4 can be done when all are
approved? Please let me know so that I can start working on those ASAP.

Thanks,
Kapil

On Tue, Sep 2, 2025 at 4:48 AM Petri Tuomola <pe...@gmail.com>
wrote:

> Thanks Kapil - but this JIRA / PRs seem to be about refactoring the
> Collection Sheet API to the new command processing pattern, not about
> fixing defects in the API.
>
> Before we start any refactoring, we should make sure we're starting from a
> known good state and that there's sufficient unit testing coverage to test
> the functionality, so we know if we've broken something during refactoring.
>
> To get to this point, can you share what your comment "The *Collection
> Sheet API* in its current state appears non-functional and contains
> several logical errors" refers to?
>
> If you can raise a JIRA for each of the logical errors (and ideally
> provide add a test case into the automated tests to demonstrate each), then
> let's first fix these errors. After the API works as expected, we can then
> proceed to refactor it.
>
> Regards
> Petri
>
>
>
> ------------------------------
> *From:* Kapil Panchal <ka...@gmail.com>
> *Sent:* Monday, September 1, 2025 7:42 PM
> *To:* dev@fineract.apache.org <de...@fineract.apache.org>
> *Cc:* Edward Cable <ed...@mifos.org>
> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
>
> Hi Petri,
>
> Please take a look at the Jira Ticket
> https://issues.apache.org/jira/browse/FINERACT-2290. I had released a
> Pull Request https://github.com/apache/fineract/pull/4943.
>
> Thanks,
> Kapil
>
> On Mon, Sep 1, 2025 at 7:27 AM Petri Tuomola <pe...@gmail.com>
> wrote:
>
> Hi
>
> I'm happy to help fix issues with Collection Sheet API.
>
> If I look at the emails that started all this, I can only see this comment:
>
> "The *Collection Sheet API* in its current state appears non-functional
> and contains several logical errors"
>
>
> I think it's fair to expect a bit more detailed bug report than that.
>
> Is there any more details on this issue: how is it non-functional, and
> what are the logical errors?
>
>
> Who can provide the list of JIRAs that need to be fixed? I can then have a
> look this weekend.
>
>
> Regards
>
> Petri
>
>
>
> ------------------------------
> *From:* James Dailey <jd...@apache.org>
> *Sent:* Saturday, August 30, 2025 3:01 AM
> *To:* dev@fineract.apache.org <de...@fineract.apache.org>; Edward Cable <
> edcable@mifos.org>
> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
>
> Devs -  Are we done with this?
>
> First, I am NOT ruling out this functionality as important but if no one
> is actually demanding it, and no one is willing to maintain it, then it
> doesn't get maintained.  That is the nature of open source projects.
>
> Second, it can happen that an outside vendor (e.g. Mifos) that relies
> on this Collection Sheet functionality (or any functionality) inadvertently
> drops that from their front end release.  Because the tests at Fineract do
> not cover this fully, no one at the Vendor and no one at the Fineract
> project will see the functionality fail in a build until a customer or
> implementer notices. So, tests are vital for ensuring ongoing
> maintainability.
>
> Third, if this type of thing isn't "naturally" part of the core - then it
> will make a lot more sense to have it be an outside extension - in which
> case the tests have to be written for the API calls, and a Vendor should
> try to get such tests contributed.   I could be wrong about this, I would
> be happy to debate where this belongs.
>
> @Edward Cable <ed...@mifos.org> - what is the assessment and status of
> needed functionality and where do you think it should live?
>
> Thanks,
> James
>
> On Mon, Aug 18, 2025 at 8:21 AM James Dailey <jd...@apache.org> wrote:
>
> Sifiso -
>
> Do we have a way to reach the tens of thousands of institutions you
> believe are there?
>
> My belief (just anecdotal information) is that it’s more like a few
> hundred institutions and that out of that number, less than 10% are on a
> recent release.  And, of that smaller number, few, if any, are still using
> Collection Sheets.  I could be wrong - I’d like the data!  I’d say that
> vendors can show up with their data here to help make this vendor neutral
> space more informed.
>
> If group lending and group-member lending and collection sheets are still
> needed, then perhaps those project volunteers can contribute the needed
> updates to keep the functionality useful.
>
> Thanks
> Thanks
>
>
> On Mon, Aug 18, 2025 at 6:48 AM Sifiso Mtetwa <si...@skyburgsystems.org>
> wrote:
>
> Hi guys,
>
>
>
> This is an interesting topic. I have been wondering why, in general, we
> seem to be deprecating more and more system functions. The individual
> collection sheet has served us well over the years and still does, If
> anything we could improve on its functionality by maybe adding a bulk
> collection sheet template with little detail compared to a full loan
> repayment bulk import template.
>
>
>
> Fineract is used by tens of thousands of organisations throughout the
> world and most of them are not on this listing and may not have a voice to
> air their concerns. Maybe we can find a way of exposing this thread to
> include more voters.
>
>
>
> Regards,
>
>
>
> *From:* James Dailey [mailto:jdailey@apache.org]
> *Sent:* Saturday, 16 August 2025 02:53
> *To:* dev@fineract.apache.org
> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
>
>
>
> Ed
>
>
>
> I agree this needs to be discussed but it is important to acknowledge that
> THIS Fineract listserv is the only official (and required) discussion
> space.  It is not the intention to bury anything in “an email thread”.
>
>
>
> When I started up Mifos we spent a lot of time looking at Collection
> Sheets and designing process flows around them. I fully know that this
> design direction was important back then in 2002-2006.
>
>
>
> However if there is no one here asking for them to be retained besides
> you, then that is a sign that they have perhaps reached an end of their
> utility.  Or, that the users are not actually here, which is a different
> problem.
>
>
>
> So,… If there is a group using collection sheets in production AND they
> are not on some permanent forked (old) version, then now is the time to
> speak up.  Here.
>
>
>
> Generally, I’m fairly certain we can refactor this with an eye toward
> extracting it from the core.  Repeating the logic in two places makes no
> sense either.  Collection sheets are kind of assembled from
> constituent loans and savings.  The balances and due payments should be
> calculated in the underlying components.
>
>
>
> As the system gets restructured we need to decide to keep this at all, to
> keep it in a new place, or as some external concept/plug in. Why wouldn’t
> we want a separate component?
>
>
>
> Cheers
>
>
>
> On Thu, Aug 14, 2025 at 4:16 PM Ed Cable <ed...@mifos.org> wrote:
>
> I'll leave the other thread for discussion of the API versioning and
> refactoring related to Collection Sheet API but wanted to create a separate
> thread regarding the deprecation of the Collection Sheet.
>
>
>
> In general, for this and removal of any functionality, it's something that
> needs to be discussed openly with the community and with a vote and not a
> decision buried in a mailing list thread.
>
>
>
> For the collection sheet specifically, more thought has to be given to its
> deprecation as the centrality and highly coupled nature of the collection
> sheet is being understated as it isn't merely a report that's to be printed
> or a form filled out via a mobile application. It's a significant portion
> of the user interface and highly coupled to many of the microfinance
> features around groups/centers/meeting scheduling, staff assignment, etc.
>
>
>
> I do agree that from a UI perspective, the collection sheet and other
> microfinance-centric functionalities and flows should be viewable based on
> a configurable setting. As it doesn't lend itself to the optimal user
> experience for a large portion of current Fineract user base.
>
>
>
> I also am supportive of a strategy of slimming Fineract to its core
> services and functionality above core Fineract services and APIs can be
> extracted out.
>
>
>
> So I do think we should give thoughtful consideration to what abstracting
> out the collection sheet and corresponding microfinance functionality would
> look like and what that effort would entail to abstract it out without
> adversely impacting the original user base of the software but it's not as
> simple as deprecating these API.
>
>
>
> I welcome others' thoughts and inputs as I know even with microfinance
> itself, the methodology has evolved and group lending and the concept of a
> collection sheet isn't as central as it once was
>
>
>
> Thanks,
>
>
>
> Ed
>
>
>
>
>
>
>
> On Wed, Aug 13, 2025 at 6:36 PM Kapil Panchal <
> kapil.panchal.development@gmail.com> wrote:
>
> Hi James,
>
>
>
> I can proceed by marking this feature as @Deprecated and/or performing a
> safe refactor to remove the API.
>
>
>
> For API versioning I agree to Aleksandar Vidakovic's proposal on adapting
> to the SpringBoot v7/Spring Framework v4. If I may, Aleksandar
> Vidakovic takes the lead on this project and I can help to support the
> conversion?
>
>
>
> Thanks,
>
> Kapil
>
>
>
> On Thu, Aug 14, 2025 at 5:07 AM James Dailey <jd...@apache.org> wrote:
>
> Hi Kapil
>
>
>
> I might suggest looking at this as an opportunity to remove the collection
> sheet entirely from the Fineract namespace.  It’s a legacy concept I and
> others designed a long time ago, originally in 2002 based on collection
> sheets we gathered from a dozen countries. It is strongly tied to concepts
> in microfinance field operations, and especially when there was no data
> connectivity.
>
>
>
> It belongs perhaps as a sort of external microservice - data loading via a
> bulk import could still be enabled.
>
> The API versioning is a good idea but needs to be more holistic across the
> platform I think.
>
>
>
>
>
>
>
> On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <ad...@gmail.com> wrote:
>
> Hi Kapil,
>
> Thank you for raising the concerns below. I’ll need some additional
> details to fully understand your points:
>
> 1.      *Collection Sheet API* – You mentioned it appears non-functional
> and contains several logical errors.
>
> o    If it’s indeed not working, that’s a separate, high-priority
> discussion.
>
> o    Could you clarify which logical errors you were referring to, and
> what specifically makes you think it’s non-functional?
>
> 2.      *Service annotations* – You noted that service methods are not
> annotated with @Service and that beans are defined manually.
>
> o    Are you referring to the
> CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being defined
> via configuration?
>
> 3.      *Repository wrappers annotated with **@Service* – You mentioned
> that this mandates full unit test coverage but that they should ideally be
> annotated with @Component.
>
> o    Could you point out the exact classes you had in mind?
>
> As for the other points, I agree we can refactor and remove redundant
> logic—please feel free to suggest specific improvements or start work on
> them immediately!
>
> However, be careful by moving anything into the fineract-core… We are
> aiming to keep it as small as possible as everything is built on top of
> this module! If collection sheet are used for loans and savings - for
> example - than the recommended move is NOT to move this logic into core!
>
> Either:
>
> - we split the logic into fineract-loan and fineract-savings
>
> - Move the logic into a new module
>
> - Leave it in fineract-provider for now
>
>
>
> Shall you have any questions, please let us know!
>
> Regards,
> Adam
>
>
>
>
>
> On 2025. Aug 11., at 12:09, Kapil Panchal <
> kapil.panchal.development@gmail.com> wrote:
>
>
>
> Hi Adam,
>
> I’m currently working on *FINERACT-2290* and have a few questions before
> I submit a pull request.
>
> The *Collection Sheet API* in its current state appears non-functional
> and contains several logical errors. It seems there was an earlier attempt
> to convert from a JSON string request parameter to a class-based request
> object, but:
>
> Certain fields are missing.
>
> The serializer is not correctly populating the objects, which causes the
> conditional checks to be bypassed and results in incorrect (false)
> responses.
>
> This change set is *high risk* because it touches most of the loan and
> savings product logic. I’ve had to refactor almost all major methods.
> Extensive integration and end-to-end testing will be required to ensure
> there are no regressions, especially in edge cases. At present, there are
> no unit or integration tests for this functionality, and test creation is
> outside the current ticket scope. I’ve been iterating on this for a while,
> and only today have I reached a stable state after several experimental and
> build-breaking attempts.
>
> *Key Observations:*
>
> Service methods are not annotated with @Service; instead, beans are
> defined manually.
>
> Repository wrappers are annotated with @Service. This mandates full unit
> test coverage for these methods, but they should ideally be annotated with
> @Component.
>
> I agree with prior discussions on separating bean validation — having a
> dedicated @Component validation class allows the request object to handle
> checks independent of database queries.
>
> Validation components can also perform database-related validations; these
> can be injected into service classes for cleaner architecture.
>
> Such validation components should be placed in *Fineract-Core* so they
> are reusable across modules, reducing future refactoring needs.
>
> The current design of having commands in *Fineract-Core* and
> handlers/services/repositories in respective modules is good — it cleanly
> decouples command definition from execution.
>
> There is extensive use of this. in singleton contexts (API, Service,
> Repository). While not harmful, it’s unnecessary boilerplate.
>
> Multiple redundant intermediate DTOs exist where the request DTO itself
> could be reused for data transfer.
>
> I found redundant logic — e.g., a for loop with a break statement that
> effectively executes only once; this can be simplified.
>
> Some JDBC template queries use reserved SQL keywords, causing exceptions.
> Refactoring these queries resolves the issue and returns proper response
> objects.
>
> *Suggestions:*
>
> *Where appropriate, large tickets should be broken into subtasks to manage
> complexity and reviewability.*
>
> It may help to have a dedicated *developer-only Slack channel *for
> technical discussions. This could complement other community spaces if
> there’s a need to keep certain conversations more focused.
>
> What are your thoughts on the above?
>
> Thanks,
> Kapil
>
>
>
>
>
>
> --
>
> *Ed Cable*
>
> President/CEO, Mifos Initiative
>
> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>
>
>
> *Collectively Creating a World of 3 Billion Maries | *http://mifos.org *
> <http://facebook.com/mifos>  <http://www.twitter.com/mifos>*
>
>
>
>

Re: Collection Sheet Deprecation was [Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)]

[Petri Tuomola <pe...@gmail.com>]

Thanks Kapil - but this JIRA / PRs seem to be about refactoring the Collection Sheet API to the new command processing pattern, not about fixing defects in the API.

Before we start any refactoring, we should make sure we're starting from a known good state and that there's sufficient unit testing coverage to test the functionality, so we know if we've broken something during refactoring.

To get to this point, can you share what your comment "The Collection Sheet API in its current state appears non-functional and contains several logical errors" refers to?

If you can raise a JIRA for each of the logical errors (and ideally provide add a test case into the automated tests to demonstrate each), then let's first fix these errors. After the API works as expected, we can then proceed to refactor it.

Regards
Petri



________________________________
From: Kapil Panchal <ka...@gmail.com>
Sent: Monday, September 1, 2025 7:42 PM
To: dev@fineract.apache.org <de...@fineract.apache.org>
Cc: Edward Cable <ed...@mifos.org>
Subject: Re: Collection Sheet Deprecation was [Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)]

Hi Petri,

Please take a look at the Jira Ticket https://issues.apache.org/jira/browse/FINERACT-2290. I had released a Pull Request https://github.com/apache/fineract/pull/4943.

Thanks,
Kapil

On Mon, Sep 1, 2025 at 7:27 AM Petri Tuomola <pe...@gmail.com>> wrote:
Hi

I'm happy to help fix issues with Collection Sheet API.

If I look at the emails that started all this, I can only see this comment:


"The Collection Sheet API in its current state appears non-functional and contains several logical errors"


I think it's fair to expect a bit more detailed bug report than that.

Is there any more details on this issue: how is it non-functional, and what are the logical errors?


Who can provide the list of JIRAs that need to be fixed? I can then have a look this weekend.


Regards

Petri



________________________________
From: James Dailey <jd...@apache.org>>
Sent: Saturday, August 30, 2025 3:01 AM
To: dev@fineract.apache.org<ma...@fineract.apache.org> <de...@fineract.apache.org>>; Edward Cable <ed...@mifos.org>>
Subject: Re: Collection Sheet Deprecation was [Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)]

Devs -  Are we done with this?

First, I am NOT ruling out this functionality as important but if no one is actually demanding it, and no one is willing to maintain it, then it doesn't get maintained.  That is the nature of open source projects.

Second, it can happen that an outside vendor (e.g. Mifos) that relies on this Collection Sheet functionality (or any functionality) inadvertently drops that from their front end release.  Because the tests at Fineract do not cover this fully, no one at the Vendor and no one at the Fineract project will see the functionality fail in a build until a customer or implementer notices. So, tests are vital for ensuring ongoing maintainability.

Third, if this type of thing isn't "naturally" part of the core - then it will make a lot more sense to have it be an outside extension - in which case the tests have to be written for the API calls, and a Vendor should try to get such tests contributed.   I could be wrong about this, I would be happy to debate where this belongs.

@Edward Cable<ma...@mifos.org> - what is the assessment and status of needed functionality and where do you think it should live?

Thanks,
James

On Mon, Aug 18, 2025 at 8:21 AM James Dailey <jd...@apache.org>> wrote:
Sifiso -

Do we have a way to reach the tens of thousands of institutions you believe are there?

My belief (just anecdotal information) is that it’s more like a few hundred institutions and that out of that number, less than 10% are on a recent release.  And, of that smaller number, few, if any, are still using Collection Sheets.  I could be wrong - I’d like the data!  I’d say that vendors can show up with their data here to help make this vendor neutral space more informed.

If group lending and group-member lending and collection sheets are still needed, then perhaps those project volunteers can contribute the needed updates to keep the functionality useful.

Thanks
Thanks


On Mon, Aug 18, 2025 at 6:48 AM Sifiso Mtetwa <si...@skyburgsystems.org>> wrote:

Hi guys,



This is an interesting topic. I have been wondering why, in general, we seem to be deprecating more and more system functions. The individual collection sheet has served us well over the years and still does, If anything we could improve on its functionality by maybe adding a bulk collection sheet template with little detail compared to a full loan repayment bulk import template.



Fineract is used by tens of thousands of organisations throughout the world and most of them are not on this listing and may not have a voice to air their concerns. Maybe we can find a way of exposing this thread to include more voters.



Regards,



From: James Dailey [mailto:jdailey@apache.org<ma...@apache.org>]
Sent: Saturday, 16 August 2025 02:53
To: dev@fineract.apache.org<ma...@fineract.apache.org>
Subject: Re: Collection Sheet Deprecation was [Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)]



Ed



I agree this needs to be discussed but it is important to acknowledge that THIS Fineract listserv is the only official (and required) discussion space.  It is not the intention to bury anything in “an email thread”.



When I started up Mifos we spent a lot of time looking at Collection Sheets and designing process flows around them. I fully know that this design direction was important back then in 2002-2006.



However if there is no one here asking for them to be retained besides you, then that is a sign that they have perhaps reached an end of their utility.  Or, that the users are not actually here, which is a different problem.



So,… If there is a group using collection sheets in production AND they are not on some permanent forked (old) version, then now is the time to speak up.  Here.



Generally, I’m fairly certain we can refactor this with an eye toward extracting it from the core.  Repeating the logic in two places makes no sense either.  Collection sheets are kind of assembled from constituent loans and savings.  The balances and due payments should be calculated in the underlying components.



As the system gets restructured we need to decide to keep this at all, to keep it in a new place, or as some external concept/plug in. Why wouldn’t we want a separate component?



Cheers



On Thu, Aug 14, 2025 at 4:16 PM Ed Cable <ed...@mifos.org>> wrote:

I'll leave the other thread for discussion of the API versioning and refactoring related to Collection Sheet API but wanted to create a separate thread regarding the deprecation of the Collection Sheet.



In general, for this and removal of any functionality, it's something that needs to be discussed openly with the community and with a vote and not a decision buried in a mailing list thread.



For the collection sheet specifically, more thought has to be given to its deprecation as the centrality and highly coupled nature of the collection sheet is being understated as it isn't merely a report that's to be printed or a form filled out via a mobile application. It's a significant portion of the user interface and highly coupled to many of the microfinance features around groups/centers/meeting scheduling, staff assignment, etc.



I do agree that from a UI perspective, the collection sheet and other microfinance-centric functionalities and flows should be viewable based on a configurable setting. As it doesn't lend itself to the optimal user experience for a large portion of current Fineract user base.



I also am supportive of a strategy of slimming Fineract to its core services and functionality above core Fineract services and APIs can be extracted out.



So I do think we should give thoughtful consideration to what abstracting out the collection sheet and corresponding microfinance functionality would look like and what that effort would entail to abstract it out without adversely impacting the original user base of the software but it's not as simple as deprecating these API.



I welcome others' thoughts and inputs as I know even with microfinance itself, the methodology has evolved and group lending and the concept of a collection sheet isn't as central as it once was



Thanks,



Ed







On Wed, Aug 13, 2025 at 6:36 PM Kapil Panchal <ka...@gmail.com>> wrote:

Hi James,



I can proceed by marking this feature as @Deprecated and/or performing a safe refactor to remove the API.



For API versioning I agree to Aleksandar Vidakovic's proposal on adapting to the SpringBoot v7/Spring Framework v4. If I may, Aleksandar Vidakovic takes the lead on this project and I can help to support the conversion?



Thanks,

Kapil



On Thu, Aug 14, 2025 at 5:07 AM James Dailey <jd...@apache.org>> wrote:

Hi Kapil



I might suggest looking at this as an opportunity to remove the collection sheet entirely from the Fineract namespace.  It’s a legacy concept I and others designed a long time ago, originally in 2002 based on collection sheets we gathered from a dozen countries. It is strongly tied to concepts in microfinance field operations, and especially when there was no data connectivity.



It belongs perhaps as a sort of external microservice - data loading via a bulk import could still be enabled.

The API versioning is a good idea but needs to be more holistic across the platform I think.







On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <ad...@gmail.com>> wrote:

Hi Kapil,

Thank you for raising the concerns below. I’ll need some additional details to fully understand your points:

1.      Collection Sheet API – You mentioned it appears non-functional and contains several logical errors.

o    If it’s indeed not working, that’s a separate, high-priority discussion.

o    Could you clarify which logical errors you were referring to, and what specifically makes you think it’s non-functional?

2.      Service annotations – You noted that service methods are not annotated with @Service and that beans are defined manually.

o    Are you referring to the CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being defined via configuration?

3.      Repository wrappers annotated with @Service – You mentioned that this mandates full unit test coverage but that they should ideally be annotated with @Component.

o    Could you point out the exact classes you had in mind?

As for the other points, I agree we can refactor and remove redundant logic—please feel free to suggest specific improvements or start work on them immediately!

However, be careful by moving anything into the fineract-core… We are aiming to keep it as small as possible as everything is built on top of this module! If collection sheet are used for loans and savings - for example - than the recommended move is NOT to move this logic into core!

Either:

- we split the logic into fineract-loan and fineract-savings

- Move the logic into a new module

- Leave it in fineract-provider for now



Shall you have any questions, please let us know!

Regards,
Adam





On 2025. Aug 11., at 12:09, Kapil Panchal <ka...@gmail.com>> wrote:



Hi Adam,

I’m currently working on FINERACT-2290 and have a few questions before I submit a pull request.

The Collection Sheet API in its current state appears non-functional and contains several logical errors. It seems there was an earlier attempt to convert from a JSON string request parameter to a class-based request object, but:

Certain fields are missing.

The serializer is not correctly populating the objects, which causes the conditional checks to be bypassed and results in incorrect (false) responses.

This change set is high risk because it touches most of the loan and savings product logic. I’ve had to refactor almost all major methods. Extensive integration and end-to-end testing will be required to ensure there are no regressions, especially in edge cases. At present, there are no unit or integration tests for this functionality, and test creation is outside the current ticket scope. I’ve been iterating on this for a while, and only today have I reached a stable state after several experimental and build-breaking attempts.

Key Observations:

Service methods are not annotated with @Service; instead, beans are defined manually.

Repository wrappers are annotated with @Service. This mandates full unit test coverage for these methods, but they should ideally be annotated with @Component.

I agree with prior discussions on separating bean validation — having a dedicated @Component validation class allows the request object to handle checks independent of database queries.

Validation components can also perform database-related validations; these can be injected into service classes for cleaner architecture.

Such validation components should be placed in Fineract-Core so they are reusable across modules, reducing future refactoring needs.

The current design of having commands in Fineract-Core and handlers/services/repositories in respective modules is good — it cleanly decouples command definition from execution.

There is extensive use of this. in singleton contexts (API, Service, Repository). While not harmful, it’s unnecessary boilerplate.

Multiple redundant intermediate DTOs exist where the request DTO itself could be reused for data transfer.

I found redundant logic — e.g., a for loop with a break statement that effectively executes only once; this can be simplified.

Some JDBC template queries use reserved SQL keywords, causing exceptions. Refactoring these queries resolves the issue and returns proper response objects.

Suggestions:

Where appropriate, large tickets should be broken into subtasks to manage complexity and reviewability.

It may help to have a dedicated developer-only Slack channel for technical discussions. This could complement other community spaces if there’s a need to keep certain conversations more focused.

What are your thoughts on the above?

Thanks,
Kapil






--

Ed Cable

President/CEO, Mifos Initiative

edcable@mifos.org<ma...@mifos.org> | Skype: edcable | Mobile: +1.484.477.8649



Collectively Creating a World of 3 Billion Maries | http://mifos.org [https://secure.plimus.com/developers/817570/Template/icon-tiny-facebook.png] <http://facebook.com/mifos>  [http://organizationsandmarkets.files.wordpress.com/2010/04/icon-tiny-twitter.png] <http://www.twitter.com/mifos>

Re: Collection Sheet Deprecation was [Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)]

[Kapil Panchal <ka...@gmail.com>]

Hi Petri,

Please take a look at the Jira Ticket
https://issues.apache.org/jira/browse/FINERACT-2290. I had released a Pull
Request https://github.com/apache/fineract/pull/4943.

Thanks,
Kapil

On Mon, Sep 1, 2025 at 7:27 AM Petri Tuomola <pe...@gmail.com>
wrote:

> Hi
>
> I'm happy to help fix issues with Collection Sheet API.
>
> If I look at the emails that started all this, I can only see this comment:
>
> "The *Collection Sheet API* in its current state appears non-functional
> and contains several logical errors"
>
>
> I think it's fair to expect a bit more detailed bug report than that.
>
> Is there any more details on this issue: how is it non-functional, and
> what are the logical errors?
>
>
> Who can provide the list of JIRAs that need to be fixed? I can then have a
> look this weekend.
>
>
> Regards
>
> Petri
>
>
>
> ------------------------------
> *From:* James Dailey <jd...@apache.org>
> *Sent:* Saturday, August 30, 2025 3:01 AM
> *To:* dev@fineract.apache.org <de...@fineract.apache.org>; Edward Cable <
> edcable@mifos.org>
> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
>
> Devs -  Are we done with this?
>
> First, I am NOT ruling out this functionality as important but if no one
> is actually demanding it, and no one is willing to maintain it, then it
> doesn't get maintained.  That is the nature of open source projects.
>
> Second, it can happen that an outside vendor (e.g. Mifos) that relies
> on this Collection Sheet functionality (or any functionality) inadvertently
> drops that from their front end release.  Because the tests at Fineract do
> not cover this fully, no one at the Vendor and no one at the Fineract
> project will see the functionality fail in a build until a customer or
> implementer notices. So, tests are vital for ensuring ongoing
> maintainability.
>
> Third, if this type of thing isn't "naturally" part of the core - then it
> will make a lot more sense to have it be an outside extension - in which
> case the tests have to be written for the API calls, and a Vendor should
> try to get such tests contributed.   I could be wrong about this, I would
> be happy to debate where this belongs.
>
> @Edward Cable <ed...@mifos.org> - what is the assessment and status of
> needed functionality and where do you think it should live?
>
> Thanks,
> James
>
> On Mon, Aug 18, 2025 at 8:21 AM James Dailey <jd...@apache.org> wrote:
>
> Sifiso -
>
> Do we have a way to reach the tens of thousands of institutions you
> believe are there?
>
> My belief (just anecdotal information) is that it’s more like a few
> hundred institutions and that out of that number, less than 10% are on a
> recent release.  And, of that smaller number, few, if any, are still using
> Collection Sheets.  I could be wrong - I’d like the data!  I’d say that
> vendors can show up with their data here to help make this vendor neutral
> space more informed.
>
> If group lending and group-member lending and collection sheets are still
> needed, then perhaps those project volunteers can contribute the needed
> updates to keep the functionality useful.
>
> Thanks
> Thanks
>
>
> On Mon, Aug 18, 2025 at 6:48 AM Sifiso Mtetwa <si...@skyburgsystems.org>
> wrote:
>
> Hi guys,
>
>
>
> This is an interesting topic. I have been wondering why, in general, we
> seem to be deprecating more and more system functions. The individual
> collection sheet has served us well over the years and still does, If
> anything we could improve on its functionality by maybe adding a bulk
> collection sheet template with little detail compared to a full loan
> repayment bulk import template.
>
>
>
> Fineract is used by tens of thousands of organisations throughout the
> world and most of them are not on this listing and may not have a voice to
> air their concerns. Maybe we can find a way of exposing this thread to
> include more voters.
>
>
>
> Regards,
>
>
>
> *From:* James Dailey [mailto:jdailey@apache.org]
> *Sent:* Saturday, 16 August 2025 02:53
> *To:* dev@fineract.apache.org
> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
>
>
>
> Ed
>
>
>
> I agree this needs to be discussed but it is important to acknowledge that
> THIS Fineract listserv is the only official (and required) discussion
> space.  It is not the intention to bury anything in “an email thread”.
>
>
>
> When I started up Mifos we spent a lot of time looking at Collection
> Sheets and designing process flows around them. I fully know that this
> design direction was important back then in 2002-2006.
>
>
>
> However if there is no one here asking for them to be retained besides
> you, then that is a sign that they have perhaps reached an end of their
> utility.  Or, that the users are not actually here, which is a different
> problem.
>
>
>
> So,… If there is a group using collection sheets in production AND they
> are not on some permanent forked (old) version, then now is the time to
> speak up.  Here.
>
>
>
> Generally, I’m fairly certain we can refactor this with an eye toward
> extracting it from the core.  Repeating the logic in two places makes no
> sense either.  Collection sheets are kind of assembled from
> constituent loans and savings.  The balances and due payments should be
> calculated in the underlying components.
>
>
>
> As the system gets restructured we need to decide to keep this at all, to
> keep it in a new place, or as some external concept/plug in. Why wouldn’t
> we want a separate component?
>
>
>
> Cheers
>
>
>
> On Thu, Aug 14, 2025 at 4:16 PM Ed Cable <ed...@mifos.org> wrote:
>
> I'll leave the other thread for discussion of the API versioning and
> refactoring related to Collection Sheet API but wanted to create a separate
> thread regarding the deprecation of the Collection Sheet.
>
>
>
> In general, for this and removal of any functionality, it's something that
> needs to be discussed openly with the community and with a vote and not a
> decision buried in a mailing list thread.
>
>
>
> For the collection sheet specifically, more thought has to be given to its
> deprecation as the centrality and highly coupled nature of the collection
> sheet is being understated as it isn't merely a report that's to be printed
> or a form filled out via a mobile application. It's a significant portion
> of the user interface and highly coupled to many of the microfinance
> features around groups/centers/meeting scheduling, staff assignment, etc.
>
>
>
> I do agree that from a UI perspective, the collection sheet and other
> microfinance-centric functionalities and flows should be viewable based on
> a configurable setting. As it doesn't lend itself to the optimal user
> experience for a large portion of current Fineract user base.
>
>
>
> I also am supportive of a strategy of slimming Fineract to its core
> services and functionality above core Fineract services and APIs can be
> extracted out.
>
>
>
> So I do think we should give thoughtful consideration to what abstracting
> out the collection sheet and corresponding microfinance functionality would
> look like and what that effort would entail to abstract it out without
> adversely impacting the original user base of the software but it's not as
> simple as deprecating these API.
>
>
>
> I welcome others' thoughts and inputs as I know even with microfinance
> itself, the methodology has evolved and group lending and the concept of a
> collection sheet isn't as central as it once was
>
>
>
> Thanks,
>
>
>
> Ed
>
>
>
>
>
>
>
> On Wed, Aug 13, 2025 at 6:36 PM Kapil Panchal <
> kapil.panchal.development@gmail.com> wrote:
>
> Hi James,
>
>
>
> I can proceed by marking this feature as @Deprecated and/or performing a
> safe refactor to remove the API.
>
>
>
> For API versioning I agree to Aleksandar Vidakovic's proposal on adapting
> to the SpringBoot v7/Spring Framework v4. If I may, Aleksandar
> Vidakovic takes the lead on this project and I can help to support the
> conversion?
>
>
>
> Thanks,
>
> Kapil
>
>
>
> On Thu, Aug 14, 2025 at 5:07 AM James Dailey <jd...@apache.org> wrote:
>
> Hi Kapil
>
>
>
> I might suggest looking at this as an opportunity to remove the collection
> sheet entirely from the Fineract namespace.  It’s a legacy concept I and
> others designed a long time ago, originally in 2002 based on collection
> sheets we gathered from a dozen countries. It is strongly tied to concepts
> in microfinance field operations, and especially when there was no data
> connectivity.
>
>
>
> It belongs perhaps as a sort of external microservice - data loading via a
> bulk import could still be enabled.
>
> The API versioning is a good idea but needs to be more holistic across the
> platform I think.
>
>
>
>
>
>
>
> On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <ad...@gmail.com> wrote:
>
> Hi Kapil,
>
> Thank you for raising the concerns below. I’ll need some additional
> details to fully understand your points:
>
> 1.      *Collection Sheet API* – You mentioned it appears non-functional
> and contains several logical errors.
>
> o    If it’s indeed not working, that’s a separate, high-priority
> discussion.
>
> o    Could you clarify which logical errors you were referring to, and
> what specifically makes you think it’s non-functional?
>
> 2.      *Service annotations* – You noted that service methods are not
> annotated with @Service and that beans are defined manually.
>
> o    Are you referring to the
> CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being defined
> via configuration?
>
> 3.      *Repository wrappers annotated with **@Service* – You mentioned
> that this mandates full unit test coverage but that they should ideally be
> annotated with @Component.
>
> o    Could you point out the exact classes you had in mind?
>
> As for the other points, I agree we can refactor and remove redundant
> logic—please feel free to suggest specific improvements or start work on
> them immediately!
>
> However, be careful by moving anything into the fineract-core… We are
> aiming to keep it as small as possible as everything is built on top of
> this module! If collection sheet are used for loans and savings - for
> example - than the recommended move is NOT to move this logic into core!
>
> Either:
>
> - we split the logic into fineract-loan and fineract-savings
>
> - Move the logic into a new module
>
> - Leave it in fineract-provider for now
>
>
>
> Shall you have any questions, please let us know!
>
> Regards,
> Adam
>
>
>
>
>
> On 2025. Aug 11., at 12:09, Kapil Panchal <
> kapil.panchal.development@gmail.com> wrote:
>
>
>
> Hi Adam,
>
> I’m currently working on *FINERACT-2290* and have a few questions before
> I submit a pull request.
>
> The *Collection Sheet API* in its current state appears non-functional
> and contains several logical errors. It seems there was an earlier attempt
> to convert from a JSON string request parameter to a class-based request
> object, but:
>
> Certain fields are missing.
>
> The serializer is not correctly populating the objects, which causes the
> conditional checks to be bypassed and results in incorrect (false)
> responses.
>
> This change set is *high risk* because it touches most of the loan and
> savings product logic. I’ve had to refactor almost all major methods.
> Extensive integration and end-to-end testing will be required to ensure
> there are no regressions, especially in edge cases. At present, there are
> no unit or integration tests for this functionality, and test creation is
> outside the current ticket scope. I’ve been iterating on this for a while,
> and only today have I reached a stable state after several experimental and
> build-breaking attempts.
>
> *Key Observations:*
>
> Service methods are not annotated with @Service; instead, beans are
> defined manually.
>
> Repository wrappers are annotated with @Service. This mandates full unit
> test coverage for these methods, but they should ideally be annotated with
> @Component.
>
> I agree with prior discussions on separating bean validation — having a
> dedicated @Component validation class allows the request object to handle
> checks independent of database queries.
>
> Validation components can also perform database-related validations; these
> can be injected into service classes for cleaner architecture.
>
> Such validation components should be placed in *Fineract-Core* so they
> are reusable across modules, reducing future refactoring needs.
>
> The current design of having commands in *Fineract-Core* and
> handlers/services/repositories in respective modules is good — it cleanly
> decouples command definition from execution.
>
> There is extensive use of this. in singleton contexts (API, Service,
> Repository). While not harmful, it’s unnecessary boilerplate.
>
> Multiple redundant intermediate DTOs exist where the request DTO itself
> could be reused for data transfer.
>
> I found redundant logic — e.g., a for loop with a break statement that
> effectively executes only once; this can be simplified.
>
> Some JDBC template queries use reserved SQL keywords, causing exceptions.
> Refactoring these queries resolves the issue and returns proper response
> objects.
>
> *Suggestions:*
>
> *Where appropriate, large tickets should be broken into subtasks to manage
> complexity and reviewability.*
>
> It may help to have a dedicated *developer-only Slack channel *for
> technical discussions. This could complement other community spaces if
> there’s a need to keep certain conversations more focused.
>
> What are your thoughts on the above?
>
> Thanks,
> Kapil
>
>
>
>
>
>
> --
>
> *Ed Cable*
>
> President/CEO, Mifos Initiative
>
> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>
>
>
> *Collectively Creating a World of 3 Billion Maries | *http://mifos.org *
> <http://facebook.com/mifos>  <http://www.twitter.com/mifos>*
>
>
>
>

Re: Collection Sheet Deprecation was [Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)]

[Petri Tuomola <pe...@gmail.com>]

Hi

I'm happy to help fix issues with Collection Sheet API.

If I look at the emails that started all this, I can only see this comment:


"The Collection Sheet API in its current state appears non-functional and contains several logical errors"


I think it's fair to expect a bit more detailed bug report than that.

Is there any more details on this issue: how is it non-functional, and what are the logical errors?


Who can provide the list of JIRAs that need to be fixed? I can then have a look this weekend.


Regards

Petri



________________________________
From: James Dailey <jd...@apache.org>
Sent: Saturday, August 30, 2025 3:01 AM
To: dev@fineract.apache.org <de...@fineract.apache.org>; Edward Cable <ed...@mifos.org>
Subject: Re: Collection Sheet Deprecation was [Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)]

Devs -  Are we done with this?

First, I am NOT ruling out this functionality as important but if no one is actually demanding it, and no one is willing to maintain it, then it doesn't get maintained.  That is the nature of open source projects.

Second, it can happen that an outside vendor (e.g. Mifos) that relies on this Collection Sheet functionality (or any functionality) inadvertently drops that from their front end release.  Because the tests at Fineract do not cover this fully, no one at the Vendor and no one at the Fineract project will see the functionality fail in a build until a customer or implementer notices. So, tests are vital for ensuring ongoing maintainability.

Third, if this type of thing isn't "naturally" part of the core - then it will make a lot more sense to have it be an outside extension - in which case the tests have to be written for the API calls, and a Vendor should try to get such tests contributed.   I could be wrong about this, I would be happy to debate where this belongs.

@Edward Cable<ma...@mifos.org> - what is the assessment and status of needed functionality and where do you think it should live?

Thanks,
James

On Mon, Aug 18, 2025 at 8:21 AM James Dailey <jd...@apache.org>> wrote:
Sifiso -

Do we have a way to reach the tens of thousands of institutions you believe are there?

My belief (just anecdotal information) is that it’s more like a few hundred institutions and that out of that number, less than 10% are on a recent release.  And, of that smaller number, few, if any, are still using Collection Sheets.  I could be wrong - I’d like the data!  I’d say that vendors can show up with their data here to help make this vendor neutral space more informed.

If group lending and group-member lending and collection sheets are still needed, then perhaps those project volunteers can contribute the needed updates to keep the functionality useful.

Thanks
Thanks


On Mon, Aug 18, 2025 at 6:48 AM Sifiso Mtetwa <si...@skyburgsystems.org>> wrote:

Hi guys,



This is an interesting topic. I have been wondering why, in general, we seem to be deprecating more and more system functions. The individual collection sheet has served us well over the years and still does, If anything we could improve on its functionality by maybe adding a bulk collection sheet template with little detail compared to a full loan repayment bulk import template.



Fineract is used by tens of thousands of organisations throughout the world and most of them are not on this listing and may not have a voice to air their concerns. Maybe we can find a way of exposing this thread to include more voters.



Regards,



From: James Dailey [mailto:jdailey@apache.org<ma...@apache.org>]
Sent: Saturday, 16 August 2025 02:53
To: dev@fineract.apache.org<ma...@fineract.apache.org>
Subject: Re: Collection Sheet Deprecation was [Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)]



Ed



I agree this needs to be discussed but it is important to acknowledge that THIS Fineract listserv is the only official (and required) discussion space.  It is not the intention to bury anything in “an email thread”.



When I started up Mifos we spent a lot of time looking at Collection Sheets and designing process flows around them. I fully know that this design direction was important back then in 2002-2006.



However if there is no one here asking for them to be retained besides you, then that is a sign that they have perhaps reached an end of their utility.  Or, that the users are not actually here, which is a different problem.



So,… If there is a group using collection sheets in production AND they are not on some permanent forked (old) version, then now is the time to speak up.  Here.



Generally, I’m fairly certain we can refactor this with an eye toward extracting it from the core.  Repeating the logic in two places makes no sense either.  Collection sheets are kind of assembled from constituent loans and savings.  The balances and due payments should be calculated in the underlying components.



As the system gets restructured we need to decide to keep this at all, to keep it in a new place, or as some external concept/plug in. Why wouldn’t we want a separate component?



Cheers



On Thu, Aug 14, 2025 at 4:16 PM Ed Cable <ed...@mifos.org>> wrote:

I'll leave the other thread for discussion of the API versioning and refactoring related to Collection Sheet API but wanted to create a separate thread regarding the deprecation of the Collection Sheet.



In general, for this and removal of any functionality, it's something that needs to be discussed openly with the community and with a vote and not a decision buried in a mailing list thread.



For the collection sheet specifically, more thought has to be given to its deprecation as the centrality and highly coupled nature of the collection sheet is being understated as it isn't merely a report that's to be printed or a form filled out via a mobile application. It's a significant portion of the user interface and highly coupled to many of the microfinance features around groups/centers/meeting scheduling, staff assignment, etc.



I do agree that from a UI perspective, the collection sheet and other microfinance-centric functionalities and flows should be viewable based on a configurable setting. As it doesn't lend itself to the optimal user experience for a large portion of current Fineract user base.



I also am supportive of a strategy of slimming Fineract to its core services and functionality above core Fineract services and APIs can be extracted out.



So I do think we should give thoughtful consideration to what abstracting out the collection sheet and corresponding microfinance functionality would look like and what that effort would entail to abstract it out without adversely impacting the original user base of the software but it's not as simple as deprecating these API.



I welcome others' thoughts and inputs as I know even with microfinance itself, the methodology has evolved and group lending and the concept of a collection sheet isn't as central as it once was



Thanks,



Ed







On Wed, Aug 13, 2025 at 6:36 PM Kapil Panchal <ka...@gmail.com>> wrote:

Hi James,



I can proceed by marking this feature as @Deprecated and/or performing a safe refactor to remove the API.



For API versioning I agree to Aleksandar Vidakovic's proposal on adapting to the SpringBoot v7/Spring Framework v4. If I may, Aleksandar Vidakovic takes the lead on this project and I can help to support the conversion?



Thanks,

Kapil



On Thu, Aug 14, 2025 at 5:07 AM James Dailey <jd...@apache.org>> wrote:

Hi Kapil



I might suggest looking at this as an opportunity to remove the collection sheet entirely from the Fineract namespace.  It’s a legacy concept I and others designed a long time ago, originally in 2002 based on collection sheets we gathered from a dozen countries. It is strongly tied to concepts in microfinance field operations, and especially when there was no data connectivity.



It belongs perhaps as a sort of external microservice - data loading via a bulk import could still be enabled.

The API versioning is a good idea but needs to be more holistic across the platform I think.







On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <ad...@gmail.com>> wrote:

Hi Kapil,

Thank you for raising the concerns below. I’ll need some additional details to fully understand your points:

1.      Collection Sheet API – You mentioned it appears non-functional and contains several logical errors.

o    If it’s indeed not working, that’s a separate, high-priority discussion.

o    Could you clarify which logical errors you were referring to, and what specifically makes you think it’s non-functional?

2.      Service annotations – You noted that service methods are not annotated with @Service and that beans are defined manually.

o    Are you referring to the CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being defined via configuration?

3.      Repository wrappers annotated with @Service – You mentioned that this mandates full unit test coverage but that they should ideally be annotated with @Component.

o    Could you point out the exact classes you had in mind?

As for the other points, I agree we can refactor and remove redundant logic—please feel free to suggest specific improvements or start work on them immediately!

However, be careful by moving anything into the fineract-core… We are aiming to keep it as small as possible as everything is built on top of this module! If collection sheet are used for loans and savings - for example - than the recommended move is NOT to move this logic into core!

Either:

- we split the logic into fineract-loan and fineract-savings

- Move the logic into a new module

- Leave it in fineract-provider for now



Shall you have any questions, please let us know!

Regards,
Adam





On 2025. Aug 11., at 12:09, Kapil Panchal <ka...@gmail.com>> wrote:



Hi Adam,

I’m currently working on FINERACT-2290 and have a few questions before I submit a pull request.

The Collection Sheet API in its current state appears non-functional and contains several logical errors. It seems there was an earlier attempt to convert from a JSON string request parameter to a class-based request object, but:

Certain fields are missing.

The serializer is not correctly populating the objects, which causes the conditional checks to be bypassed and results in incorrect (false) responses.

This change set is high risk because it touches most of the loan and savings product logic. I’ve had to refactor almost all major methods. Extensive integration and end-to-end testing will be required to ensure there are no regressions, especially in edge cases. At present, there are no unit or integration tests for this functionality, and test creation is outside the current ticket scope. I’ve been iterating on this for a while, and only today have I reached a stable state after several experimental and build-breaking attempts.

Key Observations:

Service methods are not annotated with @Service; instead, beans are defined manually.

Repository wrappers are annotated with @Service. This mandates full unit test coverage for these methods, but they should ideally be annotated with @Component.

I agree with prior discussions on separating bean validation — having a dedicated @Component validation class allows the request object to handle checks independent of database queries.

Validation components can also perform database-related validations; these can be injected into service classes for cleaner architecture.

Such validation components should be placed in Fineract-Core so they are reusable across modules, reducing future refactoring needs.

The current design of having commands in Fineract-Core and handlers/services/repositories in respective modules is good — it cleanly decouples command definition from execution.

There is extensive use of this. in singleton contexts (API, Service, Repository). While not harmful, it’s unnecessary boilerplate.

Multiple redundant intermediate DTOs exist where the request DTO itself could be reused for data transfer.

I found redundant logic — e.g., a for loop with a break statement that effectively executes only once; this can be simplified.

Some JDBC template queries use reserved SQL keywords, causing exceptions. Refactoring these queries resolves the issue and returns proper response objects.

Suggestions:

Where appropriate, large tickets should be broken into subtasks to manage complexity and reviewability.

It may help to have a dedicated developer-only Slack channel for technical discussions. This could complement other community spaces if there’s a need to keep certain conversations more focused.

What are your thoughts on the above?

Thanks,
Kapil






--

Ed Cable

President/CEO, Mifos Initiative

edcable@mifos.org<ma...@mifos.org> | Skype: edcable | Mobile: +1.484.477.8649



Collectively Creating a World of 3 Billion Maries | http://mifos.org [https://secure.plimus.com/developers/817570/Template/icon-tiny-facebook.png] <http://facebook.com/mifos>  [http://organizationsandmarkets.files.wordpress.com/2010/04/icon-tiny-twitter.png] <http://www.twitter.com/mifos>

Re: Collection Sheet Deprecation was [Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)]

[James Dailey <jd...@apache.org>]

Devs -  Are we done with this?

First, I am NOT ruling out this functionality as important but if no one is
actually demanding it, and no one is willing to maintain it, then it
doesn't get maintained.  That is the nature of open source projects.

Second, it can happen that an outside vendor (e.g. Mifos) that relies
on this Collection Sheet functionality (or any functionality) inadvertently
drops that from their front end release.  Because the tests at Fineract do
not cover this fully, no one at the Vendor and no one at the Fineract
project will see the functionality fail in a build until a customer or
implementer notices. So, tests are vital for ensuring ongoing
maintainability.

Third, if this type of thing isn't "naturally" part of the core - then it
will make a lot more sense to have it be an outside extension - in which
case the tests have to be written for the API calls, and a Vendor should
try to get such tests contributed.   I could be wrong about this, I would
be happy to debate where this belongs.

@Edward Cable <ed...@mifos.org> - what is the assessment and status of
needed functionality and where do you think it should live?

Thanks,
James

On Mon, Aug 18, 2025 at 8:21 AM James Dailey <jd...@apache.org> wrote:

> Sifiso -
>
> Do we have a way to reach the tens of thousands of institutions you
> believe are there?
>
> My belief (just anecdotal information) is that it’s more like a few
> hundred institutions and that out of that number, less than 10% are on a
> recent release.  And, of that smaller number, few, if any, are still using
> Collection Sheets.  I could be wrong - I’d like the data!  I’d say that
> vendors can show up with their data here to help make this vendor neutral
> space more informed.
>
> If group lending and group-member lending and collection sheets are still
> needed, then perhaps those project volunteers can contribute the needed
> updates to keep the functionality useful.
>
> Thanks
> Thanks
>
>
> On Mon, Aug 18, 2025 at 6:48 AM Sifiso Mtetwa <si...@skyburgsystems.org>
> wrote:
>
>> Hi guys,
>>
>>
>>
>> This is an interesting topic. I have been wondering why, in general, we
>> seem to be deprecating more and more system functions. The individual
>> collection sheet has served us well over the years and still does, If
>> anything we could improve on its functionality by maybe adding a bulk
>> collection sheet template with little detail compared to a full loan
>> repayment bulk import template.
>>
>>
>>
>> Fineract is used by tens of thousands of organisations throughout the
>> world and most of them are not on this listing and may not have a voice to
>> air their concerns. Maybe we can find a way of exposing this thread to
>> include more voters.
>>
>>
>>
>> Regards,
>>
>>
>>
>> *From:* James Dailey [mailto:jdailey@apache.org]
>> *Sent:* Saturday, 16 August 2025 02:53
>> *To:* dev@fineract.apache.org
>> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
>> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
>>
>>
>>
>> Ed
>>
>>
>>
>> I agree this needs to be discussed but it is important to acknowledge
>> that THIS Fineract listserv is the only official (and required) discussion
>> space.  It is not the intention to bury anything in “an email thread”.
>>
>>
>>
>> When I started up Mifos we spent a lot of time looking at Collection
>> Sheets and designing process flows around them. I fully know that this
>> design direction was important back then in 2002-2006.
>>
>>
>>
>> However if there is no one here asking for them to be retained besides
>> you, then that is a sign that they have perhaps reached an end of their
>> utility.  Or, that the users are not actually here, which is a different
>> problem.
>>
>>
>>
>> So,… If there is a group using collection sheets in production AND they
>> are not on some permanent forked (old) version, then now is the time to
>> speak up.  Here.
>>
>>
>>
>> Generally, I’m fairly certain we can refactor this with an eye toward
>> extracting it from the core.  Repeating the logic in two places makes no
>> sense either.  Collection sheets are kind of assembled from
>> constituent loans and savings.  The balances and due payments should be
>> calculated in the underlying components.
>>
>>
>>
>> As the system gets restructured we need to decide to keep this at all, to
>> keep it in a new place, or as some external concept/plug in. Why wouldn’t
>> we want a separate component?
>>
>>
>>
>> Cheers
>>
>>
>>
>> On Thu, Aug 14, 2025 at 4:16 PM Ed Cable <ed...@mifos.org> wrote:
>>
>> I'll leave the other thread for discussion of the API versioning and
>> refactoring related to Collection Sheet API but wanted to create a separate
>> thread regarding the deprecation of the Collection Sheet.
>>
>>
>>
>> In general, for this and removal of any functionality, it's something
>> that needs to be discussed openly with the community and with a vote and
>> not a decision buried in a mailing list thread.
>>
>>
>>
>> For the collection sheet specifically, more thought has to be given to
>> its deprecation as the centrality and highly coupled nature of the
>> collection sheet is being understated as it isn't merely a report that's to
>> be printed or a form filled out via a mobile application. It's a
>> significant portion of the user interface and highly coupled to many of the
>> microfinance features around groups/centers/meeting scheduling, staff
>> assignment, etc.
>>
>>
>>
>> I do agree that from a UI perspective, the collection sheet and other
>> microfinance-centric functionalities and flows should be viewable based on
>> a configurable setting. As it doesn't lend itself to the optimal user
>> experience for a large portion of current Fineract user base.
>>
>>
>>
>> I also am supportive of a strategy of slimming Fineract to its core
>> services and functionality above core Fineract services and APIs can be
>> extracted out.
>>
>>
>>
>> So I do think we should give thoughtful consideration to what abstracting
>> out the collection sheet and corresponding microfinance functionality would
>> look like and what that effort would entail to abstract it out without
>> adversely impacting the original user base of the software but it's not as
>> simple as deprecating these API.
>>
>>
>>
>> I welcome others' thoughts and inputs as I know even with microfinance
>> itself, the methodology has evolved and group lending and the concept of a
>> collection sheet isn't as central as it once was
>>
>>
>>
>> Thanks,
>>
>>
>>
>> Ed
>>
>>
>>
>>
>>
>>
>>
>> On Wed, Aug 13, 2025 at 6:36 PM Kapil Panchal <
>> kapil.panchal.development@gmail.com> wrote:
>>
>> Hi James,
>>
>>
>>
>> I can proceed by marking this feature as @Deprecated and/or performing a
>> safe refactor to remove the API.
>>
>>
>>
>> For API versioning I agree to Aleksandar Vidakovic's proposal on adapting
>> to the SpringBoot v7/Spring Framework v4. If I may, Aleksandar
>> Vidakovic takes the lead on this project and I can help to support the
>> conversion?
>>
>>
>>
>> Thanks,
>>
>> Kapil
>>
>>
>>
>> On Thu, Aug 14, 2025 at 5:07 AM James Dailey <jd...@apache.org> wrote:
>>
>> Hi Kapil
>>
>>
>>
>> I might suggest looking at this as an opportunity to remove the
>> collection sheet entirely from the Fineract namespace.  It’s a legacy
>> concept I and others designed a long time ago, originally in 2002 based on
>> collection sheets we gathered from a dozen countries. It is strongly tied
>> to concepts in microfinance field operations, and especially when there was
>> no data connectivity.
>>
>>
>>
>> It belongs perhaps as a sort of external microservice - data loading via
>> a bulk import could still be enabled.
>>
>> The API versioning is a good idea but needs to be more holistic across
>> the platform I think.
>>
>>
>>
>>
>>
>>
>>
>> On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <ad...@gmail.com> wrote:
>>
>> Hi Kapil,
>>
>> Thank you for raising the concerns below. I’ll need some additional
>> details to fully understand your points:
>>
>> 1.      *Collection Sheet API* – You mentioned it appears non-functional
>> and contains several logical errors.
>>
>> o    If it’s indeed not working, that’s a separate, high-priority
>> discussion.
>>
>> o    Could you clarify which logical errors you were referring to, and
>> what specifically makes you think it’s non-functional?
>>
>> 2.      *Service annotations* – You noted that service methods are not
>> annotated with @Service and that beans are defined manually.
>>
>> o    Are you referring to the
>> CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being defined
>> via configuration?
>>
>> 3.      *Repository wrappers annotated with **@Service* – You mentioned
>> that this mandates full unit test coverage but that they should ideally be
>> annotated with @Component.
>>
>> o    Could you point out the exact classes you had in mind?
>>
>> As for the other points, I agree we can refactor and remove redundant
>> logic—please feel free to suggest specific improvements or start work on
>> them immediately!
>>
>> However, be careful by moving anything into the fineract-core… We are
>> aiming to keep it as small as possible as everything is built on top of
>> this module! If collection sheet are used for loans and savings - for
>> example - than the recommended move is NOT to move this logic into core!
>>
>> Either:
>>
>> - we split the logic into fineract-loan and fineract-savings
>>
>> - Move the logic into a new module
>>
>> - Leave it in fineract-provider for now
>>
>>
>>
>> Shall you have any questions, please let us know!
>>
>> Regards,
>> Adam
>>
>>
>>
>>
>>
>> On 2025. Aug 11., at 12:09, Kapil Panchal <
>> kapil.panchal.development@gmail.com> wrote:
>>
>>
>>
>> Hi Adam,
>>
>> I’m currently working on *FINERACT-2290* and have a few questions before
>> I submit a pull request.
>>
>> The *Collection Sheet API* in its current state appears non-functional
>> and contains several logical errors. It seems there was an earlier attempt
>> to convert from a JSON string request parameter to a class-based request
>> object, but:
>>
>> Certain fields are missing.
>>
>> The serializer is not correctly populating the objects, which causes the
>> conditional checks to be bypassed and results in incorrect (false)
>> responses.
>>
>> This change set is *high risk* because it touches most of the loan and
>> savings product logic. I’ve had to refactor almost all major methods.
>> Extensive integration and end-to-end testing will be required to ensure
>> there are no regressions, especially in edge cases. At present, there are
>> no unit or integration tests for this functionality, and test creation is
>> outside the current ticket scope. I’ve been iterating on this for a while,
>> and only today have I reached a stable state after several experimental and
>> build-breaking attempts.
>>
>> *Key Observations:*
>>
>> Service methods are not annotated with @Service; instead, beans are
>> defined manually.
>>
>> Repository wrappers are annotated with @Service. This mandates full unit
>> test coverage for these methods, but they should ideally be annotated with
>> @Component.
>>
>> I agree with prior discussions on separating bean validation — having a
>> dedicated @Component validation class allows the request object to handle
>> checks independent of database queries.
>>
>> Validation components can also perform database-related validations;
>> these can be injected into service classes for cleaner architecture.
>>
>> Such validation components should be placed in *Fineract-Core* so they
>> are reusable across modules, reducing future refactoring needs.
>>
>> The current design of having commands in *Fineract-Core* and
>> handlers/services/repositories in respective modules is good — it cleanly
>> decouples command definition from execution.
>>
>> There is extensive use of this. in singleton contexts (API, Service,
>> Repository). While not harmful, it’s unnecessary boilerplate.
>>
>> Multiple redundant intermediate DTOs exist where the request DTO itself
>> could be reused for data transfer.
>>
>> I found redundant logic — e.g., a for loop with a break statement that
>> effectively executes only once; this can be simplified.
>>
>> Some JDBC template queries use reserved SQL keywords, causing exceptions.
>> Refactoring these queries resolves the issue and returns proper response
>> objects.
>>
>> *Suggestions:*
>>
>> *Where appropriate, large tickets should be broken into subtasks to
>> manage complexity and reviewability.*
>>
>> It may help to have a dedicated *developer-only Slack channel *for
>> technical discussions. This could complement other community spaces if
>> there’s a need to keep certain conversations more focused.
>>
>> What are your thoughts on the above?
>>
>> Thanks,
>> Kapil
>>
>>
>>
>>
>>
>>
>> --
>>
>> *Ed Cable*
>>
>> President/CEO, Mifos Initiative
>>
>> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>>
>>
>>
>> *Collectively Creating a World of 3 Billion Maries | *http://mifos.org
>> <http://facebook.com/mifos>  <http://www.twitter.com/mifos>
>>
>>
>>
>>

Re: Collection Sheet Deprecation was [Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)]

[James Dailey <jd...@apache.org>]

Sifiso -

Do we have a way to reach the tens of thousands of institutions you believe
are there?

My belief (just anecdotal information) is that it’s more like a few hundred
institutions and that out of that number, less than 10% are on a recent
release.  And, of that smaller number, few, if any, are still using
Collection Sheets.  I could be wrong - I’d like the data!  I’d say that
vendors can show up with their data here to help make this vendor neutral
space more informed.

If group lending and group-member lending and collection sheets are still
needed, then perhaps those project volunteers can contribute the needed
updates to keep the functionality useful.

Thanks
Thanks


On Mon, Aug 18, 2025 at 6:48 AM Sifiso Mtetwa <si...@skyburgsystems.org>
wrote:

> Hi guys,
>
>
>
> This is an interesting topic. I have been wondering why, in general, we
> seem to be deprecating more and more system functions. The individual
> collection sheet has served us well over the years and still does, If
> anything we could improve on its functionality by maybe adding a bulk
> collection sheet template with little detail compared to a full loan
> repayment bulk import template.
>
>
>
> Fineract is used by tens of thousands of organisations throughout the
> world and most of them are not on this listing and may not have a voice to
> air their concerns. Maybe we can find a way of exposing this thread to
> include more voters.
>
>
>
> Regards,
>
>
>
> *From:* James Dailey [mailto:jdailey@apache.org]
> *Sent:* Saturday, 16 August 2025 02:53
> *To:* dev@fineract.apache.org
> *Subject:* Re: Collection Sheet Deprecation was [Re: Questions and
> Observations on FINERACT-2290 (Collection Sheet API Refactor)]
>
>
>
> Ed
>
>
>
> I agree this needs to be discussed but it is important to acknowledge that
> THIS Fineract listserv is the only official (and required) discussion
> space.  It is not the intention to bury anything in “an email thread”.
>
>
>
> When I started up Mifos we spent a lot of time looking at Collection
> Sheets and designing process flows around them. I fully know that this
> design direction was important back then in 2002-2006.
>
>
>
> However if there is no one here asking for them to be retained besides
> you, then that is a sign that they have perhaps reached an end of their
> utility.  Or, that the users are not actually here, which is a different
> problem.
>
>
>
> So,… If there is a group using collection sheets in production AND they
> are not on some permanent forked (old) version, then now is the time to
> speak up.  Here.
>
>
>
> Generally, I’m fairly certain we can refactor this with an eye toward
> extracting it from the core.  Repeating the logic in two places makes no
> sense either.  Collection sheets are kind of assembled from
> constituent loans and savings.  The balances and due payments should be
> calculated in the underlying components.
>
>
>
> As the system gets restructured we need to decide to keep this at all, to
> keep it in a new place, or as some external concept/plug in. Why wouldn’t
> we want a separate component?
>
>
>
> Cheers
>
>
>
> On Thu, Aug 14, 2025 at 4:16 PM Ed Cable <ed...@mifos.org> wrote:
>
> I'll leave the other thread for discussion of the API versioning and
> refactoring related to Collection Sheet API but wanted to create a separate
> thread regarding the deprecation of the Collection Sheet.
>
>
>
> In general, for this and removal of any functionality, it's something that
> needs to be discussed openly with the community and with a vote and not a
> decision buried in a mailing list thread.
>
>
>
> For the collection sheet specifically, more thought has to be given to its
> deprecation as the centrality and highly coupled nature of the collection
> sheet is being understated as it isn't merely a report that's to be printed
> or a form filled out via a mobile application. It's a significant portion
> of the user interface and highly coupled to many of the microfinance
> features around groups/centers/meeting scheduling, staff assignment, etc.
>
>
>
> I do agree that from a UI perspective, the collection sheet and other
> microfinance-centric functionalities and flows should be viewable based on
> a configurable setting. As it doesn't lend itself to the optimal user
> experience for a large portion of current Fineract user base.
>
>
>
> I also am supportive of a strategy of slimming Fineract to its core
> services and functionality above core Fineract services and APIs can be
> extracted out.
>
>
>
> So I do think we should give thoughtful consideration to what abstracting
> out the collection sheet and corresponding microfinance functionality would
> look like and what that effort would entail to abstract it out without
> adversely impacting the original user base of the software but it's not as
> simple as deprecating these API.
>
>
>
> I welcome others' thoughts and inputs as I know even with microfinance
> itself, the methodology has evolved and group lending and the concept of a
> collection sheet isn't as central as it once was
>
>
>
> Thanks,
>
>
>
> Ed
>
>
>
>
>
>
>
> On Wed, Aug 13, 2025 at 6:36 PM Kapil Panchal <
> kapil.panchal.development@gmail.com> wrote:
>
> Hi James,
>
>
>
> I can proceed by marking this feature as @Deprecated and/or performing a
> safe refactor to remove the API.
>
>
>
> For API versioning I agree to Aleksandar Vidakovic's proposal on adapting
> to the SpringBoot v7/Spring Framework v4. If I may, Aleksandar
> Vidakovic takes the lead on this project and I can help to support the
> conversion?
>
>
>
> Thanks,
>
> Kapil
>
>
>
> On Thu, Aug 14, 2025 at 5:07 AM James Dailey <jd...@apache.org> wrote:
>
> Hi Kapil
>
>
>
> I might suggest looking at this as an opportunity to remove the collection
> sheet entirely from the Fineract namespace.  It’s a legacy concept I and
> others designed a long time ago, originally in 2002 based on collection
> sheets we gathered from a dozen countries. It is strongly tied to concepts
> in microfinance field operations, and especially when there was no data
> connectivity.
>
>
>
> It belongs perhaps as a sort of external microservice - data loading via a
> bulk import could still be enabled.
>
> The API versioning is a good idea but needs to be more holistic across the
> platform I think.
>
>
>
>
>
>
>
> On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <ad...@gmail.com> wrote:
>
> Hi Kapil,
>
> Thank you for raising the concerns below. I’ll need some additional
> details to fully understand your points:
>
> 1.      *Collection Sheet API* – You mentioned it appears non-functional
> and contains several logical errors.
>
> o    If it’s indeed not working, that’s a separate, high-priority
> discussion.
>
> o    Could you clarify which logical errors you were referring to, and
> what specifically makes you think it’s non-functional?
>
> 2.      *Service annotations* – You noted that service methods are not
> annotated with @Service and that beans are defined manually.
>
> o    Are you referring to the
> CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being defined
> via configuration?
>
> 3.      *Repository wrappers annotated with **@Service* – You mentioned
> that this mandates full unit test coverage but that they should ideally be
> annotated with @Component.
>
> o    Could you point out the exact classes you had in mind?
>
> As for the other points, I agree we can refactor and remove redundant
> logic—please feel free to suggest specific improvements or start work on
> them immediately!
>
> However, be careful by moving anything into the fineract-core… We are
> aiming to keep it as small as possible as everything is built on top of
> this module! If collection sheet are used for loans and savings - for
> example - than the recommended move is NOT to move this logic into core!
>
> Either:
>
> - we split the logic into fineract-loan and fineract-savings
>
> - Move the logic into a new module
>
> - Leave it in fineract-provider for now
>
>
>
> Shall you have any questions, please let us know!
>
> Regards,
> Adam
>
>
>
>
>
> On 2025. Aug 11., at 12:09, Kapil Panchal <
> kapil.panchal.development@gmail.com> wrote:
>
>
>
> Hi Adam,
>
> I’m currently working on *FINERACT-2290* and have a few questions before
> I submit a pull request.
>
> The *Collection Sheet API* in its current state appears non-functional
> and contains several logical errors. It seems there was an earlier attempt
> to convert from a JSON string request parameter to a class-based request
> object, but:
>
> Certain fields are missing.
>
> The serializer is not correctly populating the objects, which causes the
> conditional checks to be bypassed and results in incorrect (false)
> responses.
>
> This change set is *high risk* because it touches most of the loan and
> savings product logic. I’ve had to refactor almost all major methods.
> Extensive integration and end-to-end testing will be required to ensure
> there are no regressions, especially in edge cases. At present, there are
> no unit or integration tests for this functionality, and test creation is
> outside the current ticket scope. I’ve been iterating on this for a while,
> and only today have I reached a stable state after several experimental and
> build-breaking attempts.
>
> *Key Observations:*
>
> Service methods are not annotated with @Service; instead, beans are
> defined manually.
>
> Repository wrappers are annotated with @Service. This mandates full unit
> test coverage for these methods, but they should ideally be annotated with
> @Component.
>
> I agree with prior discussions on separating bean validation — having a
> dedicated @Component validation class allows the request object to handle
> checks independent of database queries.
>
> Validation components can also perform database-related validations; these
> can be injected into service classes for cleaner architecture.
>
> Such validation components should be placed in *Fineract-Core* so they
> are reusable across modules, reducing future refactoring needs.
>
> The current design of having commands in *Fineract-Core* and
> handlers/services/repositories in respective modules is good — it cleanly
> decouples command definition from execution.
>
> There is extensive use of this. in singleton contexts (API, Service,
> Repository). While not harmful, it’s unnecessary boilerplate.
>
> Multiple redundant intermediate DTOs exist where the request DTO itself
> could be reused for data transfer.
>
> I found redundant logic — e.g., a for loop with a break statement that
> effectively executes only once; this can be simplified.
>
> Some JDBC template queries use reserved SQL keywords, causing exceptions.
> Refactoring these queries resolves the issue and returns proper response
> objects.
>
> *Suggestions:*
>
> *Where appropriate, large tickets should be broken into subtasks to manage
> complexity and reviewability.*
>
> It may help to have a dedicated *developer-only Slack channel *for
> technical discussions. This could complement other community spaces if
> there’s a need to keep certain conversations more focused.
>
> What are your thoughts on the above?
>
> Thanks,
> Kapil
>
>
>
>
>
>
> --
>
> *Ed Cable*
>
> President/CEO, Mifos Initiative
>
> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>
>
>
> *Collectively Creating a World of 3 Billion Maries | *http://mifos.org
> <http://facebook.com/mifos>  <http://www.twitter.com/mifos>
>
>
>
>

RE: Collection Sheet Deprecation was [Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)]

[Sifiso Mtetwa <si...@skyburgsystems.org>]

Hi guys,

 

This is an interesting topic. I have been wondering why, in general, we seem to be deprecating more and more system functions. The individual collection sheet has served us well over the years and still does, If anything we could improve on its functionality by maybe adding a bulk collection sheet template with little detail compared to a full loan repayment bulk import template. 

 

Fineract is used by tens of thousands of organisations throughout the world and most of them are not on this listing and may not have a voice to air their concerns. Maybe we can find a way of exposing this thread to include more voters.

 

Regards,

 

From: James Dailey [mailto:jdailey@apache.org] 
Sent: Saturday, 16 August 2025 02:53
To: dev@fineract.apache.org
Subject: Re: Collection Sheet Deprecation was [Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)]

 

Ed

 

I agree this needs to be discussed but it is important to acknowledge that THIS Fineract listserv is the only official (and required) discussion space.  It is not the intention to bury anything in “an email thread”.  

 

When I started up Mifos we spent a lot of time looking at Collection Sheets and designing process flows around them. I fully know that this design direction was important back then in 2002-2006.  

 

However if there is no one here asking for them to be retained besides you, then that is a sign that they have perhaps reached an end of their utility.  Or, that the users are not actually here, which is a different problem.  

 

So,… If there is a group using collection sheets in production AND they are not on some permanent forked (old) version, then now is the time to speak up.  Here.  

 

Generally, I’m fairly certain we can refactor this with an eye toward extracting it from the core.  Repeating the logic in two places makes no sense either.  Collection sheets are kind of assembled from constituent loans and savings.  The balances and due payments should be calculated in the underlying components.  

 

As the system gets restructured we need to decide to keep this at all, to keep it in a new place, or as some external concept/plug in. Why wouldn’t we want a separate component?  

 

Cheers 

 

On Thu, Aug 14, 2025 at 4:16 PM Ed Cable <edcable@mifos.org <ma...@mifos.org> > wrote:

I'll leave the other thread for discussion of the API versioning and refactoring related to Collection Sheet API but wanted to create a separate thread regarding the deprecation of the Collection Sheet.

 

In general, for this and removal of any functionality, it's something that needs to be discussed openly with the community and with a vote and not a decision buried in a mailing list thread.

 

For the collection sheet specifically, more thought has to be given to its deprecation as the centrality and highly coupled nature of the collection sheet is being understated as it isn't merely a report that's to be printed or a form filled out via a mobile application. It's a significant portion of the user interface and highly coupled to many of the microfinance features around groups/centers/meeting scheduling, staff assignment, etc. 

 

I do agree that from a UI perspective, the collection sheet and other microfinance-centric functionalities and flows should be viewable based on a configurable setting. As it doesn't lend itself to the optimal user experience for a large portion of current Fineract user base. 

 

I also am supportive of a strategy of slimming Fineract to its core services and functionality above core Fineract services and APIs can be extracted out. 

 

So I do think we should give thoughtful consideration to what abstracting out the collection sheet and corresponding microfinance functionality would look like and what that effort would entail to abstract it out without adversely impacting the original user base of the software but it's not as simple as deprecating these API.

 

I welcome others' thoughts and inputs as I know even with microfinance itself, the methodology has evolved and group lending and the concept of a collection sheet isn't as central as it once was

 

Thanks,

 

Ed

 

 

 

On Wed, Aug 13, 2025 at 6:36 PM Kapil Panchal <kapil.panchal.development@gmail.com <ma...@gmail.com> > wrote:

Hi James,

 

I can proceed by marking this feature as @Deprecated and/or performing a safe refactor to remove the API.

 

For API versioning I agree to Aleksandar Vidakovic's proposal on adapting to the SpringBoot v7/Spring Framework v4. If I may, Aleksandar Vidakovic takes the lead on this project and I can help to support the conversion?

 

Thanks,

Kapil

 

On Thu, Aug 14, 2025 at 5:07 AM James Dailey <jdailey@apache.org <ma...@apache.org> > wrote:

Hi Kapil 

 

I might suggest looking at this as an opportunity to remove the collection sheet entirely from the Fineract namespace.  It’s a legacy concept I and others designed a long time ago, originally in 2002 based on collection sheets we gathered from a dozen countries. It is strongly tied to concepts in microfinance field operations, and especially when there was no data connectivity. 

 

It belongs perhaps as a sort of external microservice - data loading via a bulk import could still be enabled.  

The API versioning is a good idea but needs to be more holistic across the platform I think. 

 

 

 

On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <adamsaghy@gmail.com <ma...@gmail.com> > wrote:

Hi Kapil,

Thank you for raising the concerns below. I’ll need some additional details to fully understand your points:

1.      Collection Sheet API – You mentioned it appears non-functional and contains several logical errors.

o    If it’s indeed not working, that’s a separate, high-priority discussion.

o    Could you clarify which logical errors you were referring to, and what specifically makes you think it’s non-functional?

2.      Service annotations – You noted that service methods are not annotated with @Service and that beans are defined manually.

o    Are you referring to the CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being defined via configuration?

3.      Repository wrappers annotated with @Service – You mentioned that this mandates full unit test coverage but that they should ideally be annotated with @Component.

o    Could you point out the exact classes you had in mind?

As for the other points, I agree we can refactor and remove redundant logic—please feel free to suggest specific improvements or start work on them immediately!

However, be careful by moving anything into the fineract-core… We are aiming to keep it as small as possible as everything is built on top of this module! If collection sheet are used for loans and savings - for example - than the recommended move is NOT to move this logic into core! 

Either:

- we split the logic into fineract-loan and fineract-savings 

- Move the logic into a new module

- Leave it in fineract-provider for now

 

Shall you have any questions, please let us know!

Regards,
Adam

 

 

On 2025. Aug 11., at 12:09, Kapil Panchal <kapil.panchal.development@gmail.com <ma...@gmail.com> > wrote:

 

Hi Adam,

I’m currently working on FINERACT-2290 and have a few questions before I submit a pull request.

The Collection Sheet API in its current state appears non-functional and contains several logical errors. It seems there was an earlier attempt to convert from a JSON string request parameter to a class-based request object, but:

Certain fields are missing.

The serializer is not correctly populating the objects, which causes the conditional checks to be bypassed and results in incorrect (false) responses.

This change set is high risk because it touches most of the loan and savings product logic. I’ve had to refactor almost all major methods. Extensive integration and end-to-end testing will be required to ensure there are no regressions, especially in edge cases. At present, there are no unit or integration tests for this functionality, and test creation is outside the current ticket scope. I’ve been iterating on this for a while, and only today have I reached a stable state after several experimental and build-breaking attempts.

Key Observations:

Service methods are not annotated with @Service; instead, beans are defined manually.

Repository wrappers are annotated with @Service. This mandates full unit test coverage for these methods, but they should ideally be annotated with @Component.

I agree with prior discussions on separating bean validation — having a dedicated @Component validation class allows the request object to handle checks independent of database queries.

Validation components can also perform database-related validations; these can be injected into service classes for cleaner architecture.

Such validation components should be placed in Fineract-Core so they are reusable across modules, reducing future refactoring needs.

The current design of having commands in Fineract-Core and handlers/services/repositories in respective modules is good — it cleanly decouples command definition from execution.

There is extensive use of this. in singleton contexts (API, Service, Repository). While not harmful, it’s unnecessary boilerplate.

Multiple redundant intermediate DTOs exist where the request DTO itself could be reused for data transfer.

I found redundant logic — e.g., a for loop with a break statement that effectively executes only once; this can be simplified.

Some JDBC template queries use reserved SQL keywords, causing exceptions. Refactoring these queries resolves the issue and returns proper response objects.

Suggestions:

Where appropriate, large tickets should be broken into subtasks to manage complexity and reviewability.

It may help to have a dedicated developer-only Slack channel for technical discussions. This could complement other community spaces if there’s a need to keep certain conversations more focused.

What are your thoughts on the above?

Thanks,
Kapil

 




 

-- 

Ed Cable

President/CEO, Mifos Initiative

edcable@mifos.org <ma...@mifos.org>  | Skype: edcable | Mobile: +1.484.477.8649

 

Collectively Creating a World of 3 Billion Maries | http://mifos.org  <http://facebook.com/mifos>   <http://www.twitter.com/mifos> 

 

Re: Collection Sheet Deprecation was [Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)]

[James Dailey <jd...@apache.org>]

Ed

I agree this needs to be discussed but it is important to acknowledge that
THIS Fineract listserv is the only official (and required) discussion
space.  It is not the intention to bury anything in “an email thread”.

When I started up Mifos we spent a lot of time looking at Collection Sheets
and designing process flows around them. I fully know that this design
direction was important back then in 2002-2006.

However if there is no one here asking for them to be retained besides you,
then that is a sign that they have perhaps reached an end of their
utility.  Or, that the users are not actually here, which is a different
problem.

So,… If there is a group using collection sheets in production AND they are
not on some permanent forked (old) version, then now is the time to speak
up.  Here.

Generally, I’m fairly certain we can refactor this with an eye toward
extracting it from the core.  Repeating the logic in two places makes no
sense either.  Collection sheets are kind of assembled from
constituent loans and savings.  The balances and due payments should be
calculated in the underlying components.

As the system gets restructured we need to decide to keep this at all, to
keep it in a new place, or as some external concept/plug in. Why wouldn’t
we want a separate component?

Cheers

On Thu, Aug 14, 2025 at 4:16 PM Ed Cable <ed...@mifos.org> wrote:

> I'll leave the other thread for discussion of the API versioning and
> refactoring related to Collection Sheet API but wanted to create a separate
> thread regarding the deprecation of the Collection Sheet.
>
> In general, for this and removal of any functionality, it's something that
> needs to be discussed openly with the community and with a vote and not a
> decision buried in a mailing list thread.
>
> For the collection sheet specifically, more thought has to be given to its
> deprecation as the centrality and highly coupled nature of the collection
> sheet is being understated as it isn't merely a report that's to be printed
> or a form filled out via a mobile application. It's a significant portion
> of the user interface and highly coupled to many of the microfinance
> features around groups/centers/meeting scheduling, staff assignment, etc.
>
> I do agree that from a UI perspective, the collection sheet and other
> microfinance-centric functionalities and flows should be viewable based on
> a configurable setting. As it doesn't lend itself to the optimal user
> experience for a large portion of current Fineract user base.
>
> I also am supportive of a strategy of slimming Fineract to its core
> services and functionality above core Fineract services and APIs can be
> extracted out.
>
> So I do think we should give thoughtful consideration to what abstracting
> out the collection sheet and corresponding microfinance functionality would
> look like and what that effort would entail to abstract it out without
> adversely impacting the original user base of the software but it's not as
> simple as deprecating these API.
>
> I welcome others' thoughts and inputs as I know even with microfinance
> itself, the methodology has evolved and group lending and the concept of a
> collection sheet isn't as central as it once was
>
> Thanks,
>
> Ed
>
>
>
> On Wed, Aug 13, 2025 at 6:36 PM Kapil Panchal <
> kapil.panchal.development@gmail.com> wrote:
>
>> Hi James,
>>
>> I can proceed by marking this feature as @Deprecated and/or performing a
>> safe refactor to remove the API.
>>
>> For API versioning I agree to Aleksandar Vidakovic's proposal on adapting
>> to the SpringBoot v7/Spring Framework v4. If I may, Aleksandar
>> Vidakovic takes the lead on this project and I can help to support the
>> conversion?
>>
>> Thanks,
>> Kapil
>>
>> On Thu, Aug 14, 2025 at 5:07 AM James Dailey <jd...@apache.org> wrote:
>>
>>> Hi Kapil
>>>
>>> I might suggest looking at this as an opportunity to remove the
>>> collection sheet entirely from the Fineract namespace.  It’s a legacy
>>> concept I and others designed a long time ago, originally in 2002 based on
>>> collection sheets we gathered from a dozen countries. It is strongly tied
>>> to concepts in microfinance field operations, and especially when there was
>>> no data connectivity.
>>>
>>> It belongs perhaps as a sort of external microservice - data loading via
>>> a bulk import could still be enabled.
>>>
>>> The API versioning is a good idea but needs to be more holistic across
>>> the platform I think.
>>>
>>>
>>>
>>> On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <ad...@gmail.com> wrote:
>>>
>>>> Hi Kapil,
>>>>
>>>> Thank you for raising the concerns below. I’ll need some additional
>>>> details to fully understand your points:
>>>>
>>>>    1.
>>>>
>>>>    *Collection Sheet API* – You mentioned it appears non-functional
>>>>    and contains several logical errors.
>>>>    -
>>>>
>>>>       If it’s indeed not working, that’s a separate, high-priority
>>>>       discussion.
>>>>       -
>>>>
>>>>       Could you clarify which logical errors you were referring to,
>>>>       and what specifically makes you think it’s non-functional?
>>>>       2.
>>>>
>>>>    *Service annotations* – You noted that service methods are not
>>>>    annotated with @Service and that beans are defined manually.
>>>>    -
>>>>
>>>>       Are you referring to the
>>>>       CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being
>>>>       defined via configuration?
>>>>       3.
>>>>
>>>>    *Repository wrappers annotated with @Service* – You mentioned that
>>>>    this mandates full unit test coverage but that they should ideally be
>>>>    annotated with @Component.
>>>>    -
>>>>
>>>>       Could you point out the exact classes you had in mind?
>>>>
>>>> As for the other points, I agree we can refactor and remove redundant
>>>> logic—please feel free to suggest specific improvements or start work on
>>>> them immediately!
>>>>
>>>> However, be careful by moving anything into the fineract-core… We are
>>>> aiming to keep it as small as possible as everything is built on top of
>>>> this module! If collection sheet are used for loans and savings - for
>>>> example - than the recommended move is NOT to move this logic into core!
>>>>
>>>> Either:
>>>>
>>>> - we split the logic into fineract-loan and fineract-savings
>>>>
>>>> - Move the logic into a new module
>>>>
>>>> - Leave it in fineract-provider for now
>>>>
>>>>
>>>> Shall you have any questions, please let us know!
>>>>
>>>> Regards,
>>>> Adam
>>>>
>>>>
>>>> On 2025. Aug 11., at 12:09, Kapil Panchal <
>>>> kapil.panchal.development@gmail.com> wrote:
>>>>
>>>> Hi Adam,
>>>>
>>>> I’m currently working on *FINERACT-2290* and have a few questions
>>>> before I submit a pull request.
>>>>
>>>> The *Collection Sheet API* in its current state appears non-functional
>>>> and contains several logical errors. It seems there was an earlier attempt
>>>> to convert from a JSON string request parameter to a class-based request
>>>> object, but:
>>>>
>>>> Certain fields are missing.
>>>>
>>>> The serializer is not correctly populating the objects, which causes
>>>> the conditional checks to be bypassed and results in incorrect (false)
>>>> responses.
>>>>
>>>> This change set is *high risk* because it touches most of the loan and
>>>> savings product logic. I’ve had to refactor almost all major methods.
>>>> Extensive integration and end-to-end testing will be required to ensure
>>>> there are no regressions, especially in edge cases. At present, there are
>>>> no unit or integration tests for this functionality, and test creation is
>>>> outside the current ticket scope. I’ve been iterating on this for a while,
>>>> and only today have I reached a stable state after several experimental and
>>>> build-breaking attempts.
>>>>
>>>> *Key Observations:*
>>>>
>>>> Service methods are not annotated with @Service; instead, beans are
>>>> defined manually.
>>>>
>>>> Repository wrappers are annotated with @Service. This mandates full
>>>> unit test coverage for these methods, but they should ideally be annotated
>>>> with @Component.
>>>>
>>>> I agree with prior discussions on separating bean validation — having a
>>>> dedicated @Component validation class allows the request object to handle
>>>> checks independent of database queries.
>>>>
>>>> Validation components can also perform database-related validations;
>>>> these can be injected into service classes for cleaner architecture.
>>>>
>>>> Such validation components should be placed in *Fineract-Core* so they
>>>> are reusable across modules, reducing future refactoring needs.
>>>>
>>>> The current design of having commands in *Fineract-Core* and
>>>> handlers/services/repositories in respective modules is good — it cleanly
>>>> decouples command definition from execution.
>>>>
>>>> There is extensive use of this. in singleton contexts (API, Service,
>>>> Repository). While not harmful, it’s unnecessary boilerplate.
>>>>
>>>> Multiple redundant intermediate DTOs exist where the request DTO itself
>>>> could be reused for data transfer.
>>>>
>>>> I found redundant logic — e.g., a for loop with a break statement that
>>>> effectively executes only once; this can be simplified.
>>>>
>>>> Some JDBC template queries use reserved SQL keywords, causing
>>>> exceptions. Refactoring these queries resolves the issue and returns proper
>>>> response objects.
>>>>
>>>> *Suggestions:*
>>>>
>>>> *Where appropriate, large tickets should be broken into subtasks to
>>>> manage complexity and reviewability.*
>>>>
>>>> It may help to have a dedicated *developer-only Slack channel *for
>>>> technical discussions. This could complement other community spaces if
>>>> there’s a need to keep certain conversations more focused.
>>>>
>>>> What are your thoughts on the above?
>>>>
>>>> Thanks,
>>>> Kapil
>>>>
>>>>
>>>>
>
> --
> *Ed Cable*
> President/CEO, Mifos Initiative
> edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649
>
> *Collectively Creating a World of 3 Billion Maries | *http://mifos.org
> <http://facebook.com/mifos>  <http://www.twitter.com/mifos>
>
>

Collection Sheet Deprecation was [Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)]

[Ed Cable <ed...@mifos.org>]

I'll leave the other thread for discussion of the API versioning and
refactoring related to Collection Sheet API but wanted to create a separate
thread regarding the deprecation of the Collection Sheet.

In general, for this and removal of any functionality, it's something that
needs to be discussed openly with the community and with a vote and not a
decision buried in a mailing list thread.

For the collection sheet specifically, more thought has to be given to its
deprecation as the centrality and highly coupled nature of the collection
sheet is being understated as it isn't merely a report that's to be printed
or a form filled out via a mobile application. It's a significant portion
of the user interface and highly coupled to many of the microfinance
features around groups/centers/meeting scheduling, staff assignment, etc.

I do agree that from a UI perspective, the collection sheet and other
microfinance-centric functionalities and flows should be viewable based on
a configurable setting. As it doesn't lend itself to the optimal user
experience for a large portion of current Fineract user base.

I also am supportive of a strategy of slimming Fineract to its core
services and functionality above core Fineract services and APIs can be
extracted out.

So I do think we should give thoughtful consideration to what abstracting
out the collection sheet and corresponding microfinance functionality would
look like and what that effort would entail to abstract it out without
adversely impacting the original user base of the software but it's not as
simple as deprecating these API.

I welcome others' thoughts and inputs as I know even with microfinance
itself, the methodology has evolved and group lending and the concept of a
collection sheet isn't as central as it once was

Thanks,

Ed



On Wed, Aug 13, 2025 at 6:36 PM Kapil Panchal <
kapil.panchal.development@gmail.com> wrote:

> Hi James,
>
> I can proceed by marking this feature as @Deprecated and/or performing a
> safe refactor to remove the API.
>
> For API versioning I agree to Aleksandar Vidakovic's proposal on adapting
> to the SpringBoot v7/Spring Framework v4. If I may, Aleksandar
> Vidakovic takes the lead on this project and I can help to support the
> conversion?
>
> Thanks,
> Kapil
>
> On Thu, Aug 14, 2025 at 5:07 AM James Dailey <jd...@apache.org> wrote:
>
>> Hi Kapil
>>
>> I might suggest looking at this as an opportunity to remove the
>> collection sheet entirely from the Fineract namespace.  It’s a legacy
>> concept I and others designed a long time ago, originally in 2002 based on
>> collection sheets we gathered from a dozen countries. It is strongly tied
>> to concepts in microfinance field operations, and especially when there was
>> no data connectivity.
>>
>> It belongs perhaps as a sort of external microservice - data loading via
>> a bulk import could still be enabled.
>>
>> The API versioning is a good idea but needs to be more holistic across
>> the platform I think.
>>
>>
>>
>> On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <ad...@gmail.com> wrote:
>>
>>> Hi Kapil,
>>>
>>> Thank you for raising the concerns below. I’ll need some additional
>>> details to fully understand your points:
>>>
>>>    1.
>>>
>>>    *Collection Sheet API* – You mentioned it appears non-functional and
>>>    contains several logical errors.
>>>    -
>>>
>>>       If it’s indeed not working, that’s a separate, high-priority
>>>       discussion.
>>>       -
>>>
>>>       Could you clarify which logical errors you were referring to, and
>>>       what specifically makes you think it’s non-functional?
>>>       2.
>>>
>>>    *Service annotations* – You noted that service methods are not
>>>    annotated with @Service and that beans are defined manually.
>>>    -
>>>
>>>       Are you referring to the
>>>       CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being
>>>       defined via configuration?
>>>       3.
>>>
>>>    *Repository wrappers annotated with @Service* – You mentioned that
>>>    this mandates full unit test coverage but that they should ideally be
>>>    annotated with @Component.
>>>    -
>>>
>>>       Could you point out the exact classes you had in mind?
>>>
>>> As for the other points, I agree we can refactor and remove redundant
>>> logic—please feel free to suggest specific improvements or start work on
>>> them immediately!
>>>
>>> However, be careful by moving anything into the fineract-core… We are
>>> aiming to keep it as small as possible as everything is built on top of
>>> this module! If collection sheet are used for loans and savings - for
>>> example - than the recommended move is NOT to move this logic into core!
>>>
>>> Either:
>>>
>>> - we split the logic into fineract-loan and fineract-savings
>>>
>>> - Move the logic into a new module
>>>
>>> - Leave it in fineract-provider for now
>>>
>>>
>>> Shall you have any questions, please let us know!
>>>
>>> Regards,
>>> Adam
>>>
>>>
>>> On 2025. Aug 11., at 12:09, Kapil Panchal <
>>> kapil.panchal.development@gmail.com> wrote:
>>>
>>> Hi Adam,
>>>
>>> I’m currently working on *FINERACT-2290* and have a few questions
>>> before I submit a pull request.
>>>
>>> The *Collection Sheet API* in its current state appears non-functional
>>> and contains several logical errors. It seems there was an earlier attempt
>>> to convert from a JSON string request parameter to a class-based request
>>> object, but:
>>>
>>> Certain fields are missing.
>>>
>>> The serializer is not correctly populating the objects, which causes the
>>> conditional checks to be bypassed and results in incorrect (false)
>>> responses.
>>>
>>> This change set is *high risk* because it touches most of the loan and
>>> savings product logic. I’ve had to refactor almost all major methods.
>>> Extensive integration and end-to-end testing will be required to ensure
>>> there are no regressions, especially in edge cases. At present, there are
>>> no unit or integration tests for this functionality, and test creation is
>>> outside the current ticket scope. I’ve been iterating on this for a while,
>>> and only today have I reached a stable state after several experimental and
>>> build-breaking attempts.
>>>
>>> *Key Observations:*
>>>
>>> Service methods are not annotated with @Service; instead, beans are
>>> defined manually.
>>>
>>> Repository wrappers are annotated with @Service. This mandates full unit
>>> test coverage for these methods, but they should ideally be annotated with
>>> @Component.
>>>
>>> I agree with prior discussions on separating bean validation — having a
>>> dedicated @Component validation class allows the request object to handle
>>> checks independent of database queries.
>>>
>>> Validation components can also perform database-related validations;
>>> these can be injected into service classes for cleaner architecture.
>>>
>>> Such validation components should be placed in *Fineract-Core* so they
>>> are reusable across modules, reducing future refactoring needs.
>>>
>>> The current design of having commands in *Fineract-Core* and
>>> handlers/services/repositories in respective modules is good — it cleanly
>>> decouples command definition from execution.
>>>
>>> There is extensive use of this. in singleton contexts (API, Service,
>>> Repository). While not harmful, it’s unnecessary boilerplate.
>>>
>>> Multiple redundant intermediate DTOs exist where the request DTO itself
>>> could be reused for data transfer.
>>>
>>> I found redundant logic — e.g., a for loop with a break statement that
>>> effectively executes only once; this can be simplified.
>>>
>>> Some JDBC template queries use reserved SQL keywords, causing
>>> exceptions. Refactoring these queries resolves the issue and returns proper
>>> response objects.
>>>
>>> *Suggestions:*
>>>
>>> *Where appropriate, large tickets should be broken into subtasks to
>>> manage complexity and reviewability.*
>>>
>>> It may help to have a dedicated *developer-only Slack channel *for
>>> technical discussions. This could complement other community spaces if
>>> there’s a need to keep certain conversations more focused.
>>>
>>> What are your thoughts on the above?
>>>
>>> Thanks,
>>> Kapil
>>>
>>>
>>>

-- 
*Ed Cable*
President/CEO, Mifos Initiative
edcable@mifos.org | Skype: edcable | Mobile: +1.484.477.8649

*Collectively Creating a World of 3 Billion Maries | *http://mifos.org
<http://facebook.com/mifos>  <http://www.twitter.com/mifos>

Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)

[Kapil Panchal <ka...@gmail.com>]

Hi James,

I can proceed by marking this feature as @Deprecated and/or performing a
safe refactor to remove the API.

For API versioning I agree to Aleksandar Vidakovic's proposal on adapting
to the SpringBoot v7/Spring Framework v4. If I may, Aleksandar
Vidakovic takes the lead on this project and I can help to support the
conversion?

Thanks,
Kapil

On Thu, Aug 14, 2025 at 5:07 AM James Dailey <jd...@apache.org> wrote:

> Hi Kapil
>
> I might suggest looking at this as an opportunity to remove the collection
> sheet entirely from the Fineract namespace.  It’s a legacy concept I and
> others designed a long time ago, originally in 2002 based on collection
> sheets we gathered from a dozen countries. It is strongly tied to concepts
> in microfinance field operations, and especially when there was no data
> connectivity.
>
> It belongs perhaps as a sort of external microservice - data loading via a
> bulk import could still be enabled.
>
> The API versioning is a good idea but needs to be more holistic across the
> platform I think.
>
>
>
> On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <ad...@gmail.com> wrote:
>
>> Hi Kapil,
>>
>> Thank you for raising the concerns below. I’ll need some additional
>> details to fully understand your points:
>>
>>    1.
>>
>>    *Collection Sheet API* – You mentioned it appears non-functional and
>>    contains several logical errors.
>>    -
>>
>>       If it’s indeed not working, that’s a separate, high-priority
>>       discussion.
>>       -
>>
>>       Could you clarify which logical errors you were referring to, and
>>       what specifically makes you think it’s non-functional?
>>       2.
>>
>>    *Service annotations* – You noted that service methods are not
>>    annotated with @Service and that beans are defined manually.
>>    -
>>
>>       Are you referring to the
>>       CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being
>>       defined via configuration?
>>       3.
>>
>>    *Repository wrappers annotated with @Service* – You mentioned that
>>    this mandates full unit test coverage but that they should ideally be
>>    annotated with @Component.
>>    -
>>
>>       Could you point out the exact classes you had in mind?
>>
>> As for the other points, I agree we can refactor and remove redundant
>> logic—please feel free to suggest specific improvements or start work on
>> them immediately!
>>
>> However, be careful by moving anything into the fineract-core… We are
>> aiming to keep it as small as possible as everything is built on top of
>> this module! If collection sheet are used for loans and savings - for
>> example - than the recommended move is NOT to move this logic into core!
>>
>> Either:
>>
>> - we split the logic into fineract-loan and fineract-savings
>>
>> - Move the logic into a new module
>>
>> - Leave it in fineract-provider for now
>>
>>
>> Shall you have any questions, please let us know!
>>
>> Regards,
>> Adam
>>
>>
>> On 2025. Aug 11., at 12:09, Kapil Panchal <
>> kapil.panchal.development@gmail.com> wrote:
>>
>> Hi Adam,
>>
>> I’m currently working on *FINERACT-2290* and have a few questions before
>> I submit a pull request.
>>
>> The *Collection Sheet API* in its current state appears non-functional
>> and contains several logical errors. It seems there was an earlier attempt
>> to convert from a JSON string request parameter to a class-based request
>> object, but:
>>
>> Certain fields are missing.
>>
>> The serializer is not correctly populating the objects, which causes the
>> conditional checks to be bypassed and results in incorrect (false)
>> responses.
>>
>> This change set is *high risk* because it touches most of the loan and
>> savings product logic. I’ve had to refactor almost all major methods.
>> Extensive integration and end-to-end testing will be required to ensure
>> there are no regressions, especially in edge cases. At present, there are
>> no unit or integration tests for this functionality, and test creation is
>> outside the current ticket scope. I’ve been iterating on this for a while,
>> and only today have I reached a stable state after several experimental and
>> build-breaking attempts.
>>
>> *Key Observations:*
>>
>> Service methods are not annotated with @Service; instead, beans are
>> defined manually.
>>
>> Repository wrappers are annotated with @Service. This mandates full unit
>> test coverage for these methods, but they should ideally be annotated with
>> @Component.
>>
>> I agree with prior discussions on separating bean validation — having a
>> dedicated @Component validation class allows the request object to handle
>> checks independent of database queries.
>>
>> Validation components can also perform database-related validations;
>> these can be injected into service classes for cleaner architecture.
>>
>> Such validation components should be placed in *Fineract-Core* so they
>> are reusable across modules, reducing future refactoring needs.
>>
>> The current design of having commands in *Fineract-Core* and
>> handlers/services/repositories in respective modules is good — it cleanly
>> decouples command definition from execution.
>>
>> There is extensive use of this. in singleton contexts (API, Service,
>> Repository). While not harmful, it’s unnecessary boilerplate.
>>
>> Multiple redundant intermediate DTOs exist where the request DTO itself
>> could be reused for data transfer.
>>
>> I found redundant logic — e.g., a for loop with a break statement that
>> effectively executes only once; this can be simplified.
>>
>> Some JDBC template queries use reserved SQL keywords, causing exceptions.
>> Refactoring these queries resolves the issue and returns proper response
>> objects.
>>
>> *Suggestions:*
>>
>> *Where appropriate, large tickets should be broken into subtasks to
>> manage complexity and reviewability.*
>>
>> It may help to have a dedicated *developer-only Slack channel *for
>> technical discussions. This could complement other community spaces if
>> there’s a need to keep certain conversations more focused.
>>
>> What are your thoughts on the above?
>>
>> Thanks,
>> Kapil
>>
>>
>>

Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)

[Paul <pc...@gmail.com>]

Agree.
Thinking out loud . . .  an approach to could be:

*Fineract provides basic core API services*, but recommends and supports a
scalable solution a dedicated API gateway for leveraging TPS (third-party
solutions) which can centralize and manage critical functions such as:

   -

   Policy Enforcement
   -

   Routing and Message Mediation
   -

   Security, including Authentication and Authorization
   -

   Throttling and Rate Limiting
   -

   API Monitoring
   -

   Versioning and rollback
   -

   Analytics
   -

   Developer Portal with Self-Service Access

*Fineract and Community Benefits *

   1.

   *Increased Contributor Velocity:* Focusing solely on a tight,
   high-functioning core, the community's limited resources are used more
   effectively, leading to faster development and a more robust and stable
   product.
   2.

   *Enhanced Appeal to New Projects:* A clean, stable, and modern core is
   more attractive to new project (and new contributors), growing the
   community and ensuring the project's long-term health.
   3.

   *Conforms to the Fineract Mission Statement. *The North Star for making
   decisions.  Every decision must support the mission or the community should
   modify the mission.

Forgive me for bringing in "Business" POV to the Dev Board, but Dev and
Business should MUST be joined at the hip and rely on each other to drive
the best of the product and the most effective prioritization.

On Wed, Aug 13, 2025 at 6:38 PM James Dailey <jd...@apache.org> wrote:

> Hi Kapil
>
> I might suggest looking at this as an opportunity to remove the collection
> sheet entirely from the Fineract namespace.  It’s a legacy concept I and
> others designed a long time ago, originally in 2002 based on collection
> sheets we gathered from a dozen countries. It is strongly tied to concepts
> in microfinance field operations, and especially when there was no data
> connectivity.
>
> It belongs perhaps as a sort of external microservice - data loading via a
> bulk import could still be enabled.
>
> The API versioning is a good idea but needs to be more holistic across the
> platform I think.
>
>
>
> On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <ad...@gmail.com> wrote:
>
>> Hi Kapil,
>>
>> Thank you for raising the concerns below. I’ll need some additional
>> details to fully understand your points:
>>
>>    1.
>>
>>    *Collection Sheet API* – You mentioned it appears non-functional and
>>    contains several logical errors.
>>    -
>>
>>       If it’s indeed not working, that’s a separate, high-priority
>>       discussion.
>>       -
>>
>>       Could you clarify which logical errors you were referring to, and
>>       what specifically makes you think it’s non-functional?
>>       2.
>>
>>    *Service annotations* – You noted that service methods are not
>>    annotated with @Service and that beans are defined manually.
>>    -
>>
>>       Are you referring to the
>>       CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being
>>       defined via configuration?
>>       3.
>>
>>    *Repository wrappers annotated with @Service* – You mentioned that
>>    this mandates full unit test coverage but that they should ideally be
>>    annotated with @Component.
>>    -
>>
>>       Could you point out the exact classes you had in mind?
>>
>> As for the other points, I agree we can refactor and remove redundant
>> logic—please feel free to suggest specific improvements or start work on
>> them immediately!
>>
>> However, be careful by moving anything into the fineract-core… We are
>> aiming to keep it as small as possible as everything is built on top of
>> this module! If collection sheet are used for loans and savings - for
>> example - than the recommended move is NOT to move this logic into core!
>>
>> Either:
>>
>> - we split the logic into fineract-loan and fineract-savings
>>
>> - Move the logic into a new module
>>
>> - Leave it in fineract-provider for now
>>
>>
>> Shall you have any questions, please let us know!
>>
>> Regards,
>> Adam
>>
>>
>> On 2025. Aug 11., at 12:09, Kapil Panchal <
>> kapil.panchal.development@gmail.com> wrote:
>>
>> Hi Adam,
>>
>> I’m currently working on *FINERACT-2290* and have a few questions before
>> I submit a pull request.
>>
>> The *Collection Sheet API* in its current state appears non-functional
>> and contains several logical errors. It seems there was an earlier attempt
>> to convert from a JSON string request parameter to a class-based request
>> object, but:
>>
>> Certain fields are missing.
>>
>> The serializer is not correctly populating the objects, which causes the
>> conditional checks to be bypassed and results in incorrect (false)
>> responses.
>>
>> This change set is *high risk* because it touches most of the loan and
>> savings product logic. I’ve had to refactor almost all major methods.
>> Extensive integration and end-to-end testing will be required to ensure
>> there are no regressions, especially in edge cases. At present, there are
>> no unit or integration tests for this functionality, and test creation is
>> outside the current ticket scope. I’ve been iterating on this for a while,
>> and only today have I reached a stable state after several experimental and
>> build-breaking attempts.
>>
>> *Key Observations:*
>>
>> Service methods are not annotated with @Service; instead, beans are
>> defined manually.
>>
>> Repository wrappers are annotated with @Service. This mandates full unit
>> test coverage for these methods, but they should ideally be annotated with
>> @Component.
>>
>> I agree with prior discussions on separating bean validation — having a
>> dedicated @Component validation class allows the request object to handle
>> checks independent of database queries.
>>
>> Validation components can also perform database-related validations;
>> these can be injected into service classes for cleaner architecture.
>>
>> Such validation components should be placed in *Fineract-Core* so they
>> are reusable across modules, reducing future refactoring needs.
>>
>> The current design of having commands in *Fineract-Core* and
>> handlers/services/repositories in respective modules is good — it cleanly
>> decouples command definition from execution.
>>
>> There is extensive use of this. in singleton contexts (API, Service,
>> Repository). While not harmful, it’s unnecessary boilerplate.
>>
>> Multiple redundant intermediate DTOs exist where the request DTO itself
>> could be reused for data transfer.
>>
>> I found redundant logic — e.g., a for loop with a break statement that
>> effectively executes only once; this can be simplified.
>>
>> Some JDBC template queries use reserved SQL keywords, causing exceptions.
>> Refactoring these queries resolves the issue and returns proper response
>> objects.
>>
>> *Suggestions:*
>>
>> *Where appropriate, large tickets should be broken into subtasks to
>> manage complexity and reviewability.*
>>
>> It may help to have a dedicated *developer-only Slack channel *for
>> technical discussions. This could complement other community spaces if
>> there’s a need to keep certain conversations more focused.
>>
>> What are your thoughts on the above?
>>
>> Thanks,
>> Kapil
>>
>>
>>

-- 
--
Paul

Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)

[James Dailey <jd...@apache.org>]

Hi Kapil

I might suggest looking at this as an opportunity to remove the collection
sheet entirely from the Fineract namespace.  It’s a legacy concept I and
others designed a long time ago, originally in 2002 based on collection
sheets we gathered from a dozen countries. It is strongly tied to concepts
in microfinance field operations, and especially when there was no data
connectivity.

It belongs perhaps as a sort of external microservice - data loading via a
bulk import could still be enabled.

The API versioning is a good idea but needs to be more holistic across the
platform I think.



On Mon, Aug 11, 2025 at 4:43 AM Ádám Sághy <ad...@gmail.com> wrote:

> Hi Kapil,
>
> Thank you for raising the concerns below. I’ll need some additional
> details to fully understand your points:
>
>    1.
>
>    *Collection Sheet API* – You mentioned it appears non-functional and
>    contains several logical errors.
>    -
>
>       If it’s indeed not working, that’s a separate, high-priority
>       discussion.
>       -
>
>       Could you clarify which logical errors you were referring to, and
>       what specifically makes you think it’s non-functional?
>       2.
>
>    *Service annotations* – You noted that service methods are not
>    annotated with @Service and that beans are defined manually.
>    -
>
>       Are you referring to the
>       CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being
>       defined via configuration?
>       3.
>
>    *Repository wrappers annotated with @Service* – You mentioned that
>    this mandates full unit test coverage but that they should ideally be
>    annotated with @Component.
>    -
>
>       Could you point out the exact classes you had in mind?
>
> As for the other points, I agree we can refactor and remove redundant
> logic—please feel free to suggest specific improvements or start work on
> them immediately!
>
> However, be careful by moving anything into the fineract-core… We are
> aiming to keep it as small as possible as everything is built on top of
> this module! If collection sheet are used for loans and savings - for
> example - than the recommended move is NOT to move this logic into core!
>
> Either:
>
> - we split the logic into fineract-loan and fineract-savings
>
> - Move the logic into a new module
>
> - Leave it in fineract-provider for now
>
>
> Shall you have any questions, please let us know!
>
> Regards,
> Adam
>
>
> On 2025. Aug 11., at 12:09, Kapil Panchal <
> kapil.panchal.development@gmail.com> wrote:
>
> Hi Adam,
>
> I’m currently working on *FINERACT-2290* and have a few questions before
> I submit a pull request.
>
> The *Collection Sheet API* in its current state appears non-functional
> and contains several logical errors. It seems there was an earlier attempt
> to convert from a JSON string request parameter to a class-based request
> object, but:
>
> Certain fields are missing.
>
> The serializer is not correctly populating the objects, which causes the
> conditional checks to be bypassed and results in incorrect (false)
> responses.
>
> This change set is *high risk* because it touches most of the loan and
> savings product logic. I’ve had to refactor almost all major methods.
> Extensive integration and end-to-end testing will be required to ensure
> there are no regressions, especially in edge cases. At present, there are
> no unit or integration tests for this functionality, and test creation is
> outside the current ticket scope. I’ve been iterating on this for a while,
> and only today have I reached a stable state after several experimental and
> build-breaking attempts.
>
> *Key Observations:*
>
> Service methods are not annotated with @Service; instead, beans are
> defined manually.
>
> Repository wrappers are annotated with @Service. This mandates full unit
> test coverage for these methods, but they should ideally be annotated with
> @Component.
>
> I agree with prior discussions on separating bean validation — having a
> dedicated @Component validation class allows the request object to handle
> checks independent of database queries.
>
> Validation components can also perform database-related validations; these
> can be injected into service classes for cleaner architecture.
>
> Such validation components should be placed in *Fineract-Core* so they
> are reusable across modules, reducing future refactoring needs.
>
> The current design of having commands in *Fineract-Core* and
> handlers/services/repositories in respective modules is good — it cleanly
> decouples command definition from execution.
>
> There is extensive use of this. in singleton contexts (API, Service,
> Repository). While not harmful, it’s unnecessary boilerplate.
>
> Multiple redundant intermediate DTOs exist where the request DTO itself
> could be reused for data transfer.
>
> I found redundant logic — e.g., a for loop with a break statement that
> effectively executes only once; this can be simplified.
>
> Some JDBC template queries use reserved SQL keywords, causing exceptions.
> Refactoring these queries resolves the issue and returns proper response
> objects.
>
> *Suggestions:*
>
> *Where appropriate, large tickets should be broken into subtasks to manage
> complexity and reviewability.*
>
> It may help to have a dedicated *developer-only Slack channel *for
> technical discussions. This could complement other community spaces if
> there’s a need to keep certain conversations more focused.
>
> What are your thoughts on the above?
>
> Thanks,
> Kapil
>
>
>

Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)

[Kapil Panchal <ka...@gmail.com>]

Hi Adam,

Following up on the Collection Sheet API.

*API Overview*
The API currently exposes two query parameters:

   1.

   *generateCollectionSheet:* GET request.
   -

      Throws an exception due to SQL errors in the JDBC statement — the
      query uses reserved SQL keywords.
      2.

   *saveCollectionSheet:* POST request.
   -

      Does not throw an exception, but the results returned are incorrect.
      3. *Fineract-Core:* Yes, I agree it has to be lightweight, but the
   commands, request and response objects are lightweight. All the logic and
   validations are handled by the respective modules including command
   handlers.

*Deserializer Issue*
In CollectionSheetBulkDisbursalCommandFromApiJsonDeserializer.java, the
current logic:

if (element.isJsonObject()) {
    if (topLevelJsonElement.has("bulkDisbursementTransactions")
        && topLevelJsonElement.get("bulkDisbursementTransactions").isJsonArray())
{
        final JsonArray array =
topLevelJsonElement.get("bulkDisbursementTransactions").getAsJsonArray();
        loanDisbursementTransactions = ...

…always returns false because it assumes "bulkDisbursementTransactions" is
a JSON array.

A working approach would be:

if (topLevelJsonElement != null && topLevelJsonElement.isJsonObject()) {
    JsonObject root = topLevelJsonElement.getAsJsonObject();

    if (root.has("bulkDisbursementTransactions")) {
        JsonElement disbursementElement =
root.get("bulkDisbursementTransactions");

        if (disbursementElement != null && disbursementElement.isJsonObject()) {
            JsonObject disbursementObj = disbursementElement.getAsJsonObject();

            if (disbursementObj.has("bulkRepaymentTransactions")) {
                JsonElement savingsElement =
disbursementObj.get("bulkRepaymentTransactions");

                if (savingsElement != null && savingsElement.isJsonArray()) {
                    JsonArray array = savingsElement.getAsJsonArray();
                    ...

This correctly navigates the object structure and returns the expected
values.

*Annotations*

   -

   *Wrapper classes annotated with @Service*: I am changing all
   the @Service to @Component
   1.

      PaymentTypeRepositoryWrapper
      2.

      LoanRepositoryWrapper
      3.

      HolidayRepositoryWrapper
      4.

      WorkingDaysRepositoryWrapper
      -

   *Service classes not annotated with @Service but defined via
   configuration beans*: I am going ahead and annotating it with @Service
   as later we can deprecate the configuration class
   1.

      PaymentDetailWritePlatformServiceJpaRepositoryImpl
      2.

      AccountAssociationsReadPlatformServiceImpl
      3.

      CollectionSheetReadPlatformServiceImpl
      4.

      CollectionSheetWritePlatformServiceJpaRepositoryImpl
      5.

      LoanDisbursementService
      6.

      LoanUtilService
      7.

      DepositAccountWritePlatformServiceJpaRepositoryImpl

*Suggestion – API Versioning for all *
https://issues.apache.org/jira/browse/FINERACT-2169 cards.
Given the scale of changes here, this may be a good opportunity to
consider *API
versioning*.
For example:

   -

   Current API: /v1/collectionsheet
   -

   New implementation: /v2/collectionsheet (using a separate API class)

This would allow us to refactor and restructure with more freedom while
keeping backward compatibility intact. This would also mean that each API
will now be rigorous with unit/integration/e2e tests.

What are your thoughts on proceeding with versioning for this change?

Thanks,
Kapil

On Mon, Aug 11, 2025 at 5:12 PM Ádám Sághy <ad...@gmail.com> wrote:

> Hi Kapil,
>
> Thank you for raising the concerns below. I’ll need some additional
> details to fully understand your points:
>
>    1.
>
>    *Collection Sheet API* – You mentioned it appears non-functional and
>    contains several logical errors.
>    -
>
>       If it’s indeed not working, that’s a separate, high-priority
>       discussion.
>       -
>
>       Could you clarify which logical errors you were referring to, and
>       what specifically makes you think it’s non-functional?
>       2.
>
>    *Service annotations* – You noted that service methods are not
>    annotated with @Service and that beans are defined manually.
>    -
>
>       Are you referring to the
>       CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being
>       defined via configuration?
>       3.
>
>    *Repository wrappers annotated with @Service* – You mentioned that
>    this mandates full unit test coverage but that they should ideally be
>    annotated with @Component.
>    -
>
>       Could you point out the exact classes you had in mind?
>
> As for the other points, I agree we can refactor and remove redundant
> logic—please feel free to suggest specific improvements or start work on
> them immediately!
>
> However, be careful by moving anything into the fineract-core… We are
> aiming to keep it as small as possible as everything is built on top of
> this module! If collection sheet are used for loans and savings - for
> example - than the recommended move is NOT to move this logic into core!
>
> Either:
>
> - we split the logic into fineract-loan and fineract-savings
>
> - Move the logic into a new module
>
> - Leave it in fineract-provider for now
>
>
> Shall you have any questions, please let us know!
>
> Regards,
> Adam
>
>
> On 2025. Aug 11., at 12:09, Kapil Panchal <
> kapil.panchal.development@gmail.com> wrote:
>
> Hi Adam,
>
> I’m currently working on *FINERACT-2290* and have a few questions before
> I submit a pull request.
>
> The *Collection Sheet API* in its current state appears non-functional
> and contains several logical errors. It seems there was an earlier attempt
> to convert from a JSON string request parameter to a class-based request
> object, but:
>
> Certain fields are missing.
>
> The serializer is not correctly populating the objects, which causes the
> conditional checks to be bypassed and results in incorrect (false)
> responses.
>
> This change set is *high risk* because it touches most of the loan and
> savings product logic. I’ve had to refactor almost all major methods.
> Extensive integration and end-to-end testing will be required to ensure
> there are no regressions, especially in edge cases. At present, there are
> no unit or integration tests for this functionality, and test creation is
> outside the current ticket scope. I’ve been iterating on this for a while,
> and only today have I reached a stable state after several experimental and
> build-breaking attempts.
>
> *Key Observations:*
>
> Service methods are not annotated with @Service; instead, beans are
> defined manually.
>
> Repository wrappers are annotated with @Service. This mandates full unit
> test coverage for these methods, but they should ideally be annotated with
> @Component.
>
> I agree with prior discussions on separating bean validation — having a
> dedicated @Component validation class allows the request object to handle
> checks independent of database queries.
>
> Validation components can also perform database-related validations; these
> can be injected into service classes for cleaner architecture.
>
> Such validation components should be placed in *Fineract-Core* so they
> are reusable across modules, reducing future refactoring needs.
>
> The current design of having commands in *Fineract-Core* and
> handlers/services/repositories in respective modules is good — it cleanly
> decouples command definition from execution.
>
> There is extensive use of this. in singleton contexts (API, Service,
> Repository). While not harmful, it’s unnecessary boilerplate.
>
> Multiple redundant intermediate DTOs exist where the request DTO itself
> could be reused for data transfer.
>
> I found redundant logic — e.g., a for loop with a break statement that
> effectively executes only once; this can be simplified.
>
> Some JDBC template queries use reserved SQL keywords, causing exceptions.
> Refactoring these queries resolves the issue and returns proper response
> objects.
>
> *Suggestions:*
>
> *Where appropriate, large tickets should be broken into subtasks to manage
> complexity and reviewability.*
>
> It may help to have a dedicated *developer-only Slack channel *for
> technical discussions. This could complement other community spaces if
> there’s a need to keep certain conversations more focused.
>
> What are your thoughts on the above?
>
> Thanks,
> Kapil
>
>
>

Re: Questions and Observations on FINERACT-2290 (Collection Sheet API Refactor)

[Ádám Sághy <ad...@gmail.com>]

Hi Kapil,

Thank you for raising the concerns below. I’ll need some additional details to fully understand your points:

Collection Sheet API – You mentioned it appears non-functional and contains several logical errors.

If it’s indeed not working, that’s a separate, high-priority discussion.

Could you clarify which logical errors you were referring to, and what specifically makes you think it’s non-functional?

Service annotations – You noted that service methods are not annotated with @Service and that beans are defined manually.

Are you referring to the CollectionSheetWritePlatformServiceJpaRepositoryImpl bean being defined via configuration?

Repository wrappers annotated with @Service – You mentioned that this mandates full unit test coverage but that they should ideally be annotated with @Component.

Could you point out the exact classes you had in mind?

As for the other points, I agree we can refactor and remove redundant logic—please feel free to suggest specific improvements or start work on them immediately!

However, be careful by moving anything into the fineract-core… We are aiming to keep it as small as possible as everything is built on top of this module! If collection sheet are used for loans and savings - for example - than the recommended move is NOT to move this logic into core! 

Either:

- we split the logic into fineract-loan and fineract-savings 

- Move the logic into a new module

- Leave it in fineract-provider for now



Shall you have any questions, please let us know!

Regards,
Adam



> On 2025. Aug 11., at 12:09, Kapil Panchal <ka...@gmail.com> wrote:
> 
> Hi Adam,
> 
> I’m currently working on FINERACT-2290 and have a few questions before I submit a pull request.
> 
> The Collection Sheet API in its current state appears non-functional and contains several logical errors. It seems there was an earlier attempt to convert from a JSON string request parameter to a class-based request object, but:
> 
> Certain fields are missing.
> 
> The serializer is not correctly populating the objects, which causes the conditional checks to be bypassed and results in incorrect (false) responses.
> 
> This change set is high risk because it touches most of the loan and savings product logic. I’ve had to refactor almost all major methods. Extensive integration and end-to-end testing will be required to ensure there are no regressions, especially in edge cases. At present, there are no unit or integration tests for this functionality, and test creation is outside the current ticket scope. I’ve been iterating on this for a while, and only today have I reached a stable state after several experimental and build-breaking attempts.
> 
> Key Observations:
> 
> Service methods are not annotated with @Service; instead, beans are defined manually.
> 
> Repository wrappers are annotated with @Service. This mandates full unit test coverage for these methods, but they should ideally be annotated with @Component.
> 
> I agree with prior discussions on separating bean validation — having a dedicated @Component validation class allows the request object to handle checks independent of database queries.
> 
> Validation components can also perform database-related validations; these can be injected into service classes for cleaner architecture.
> 
> Such validation components should be placed in Fineract-Core so they are reusable across modules, reducing future refactoring needs.
> 
> The current design of having commands in Fineract-Core and handlers/services/repositories in respective modules is good — it cleanly decouples command definition from execution.
> 
> There is extensive use of this. in singleton contexts (API, Service, Repository). While not harmful, it’s unnecessary boilerplate.
> 
> Multiple redundant intermediate DTOs exist where the request DTO itself could be reused for data transfer.
> 
> I found redundant logic — e.g., a for loop with a break statement that effectively executes only once; this can be simplified.
> 
> Some JDBC template queries use reserved SQL keywords, causing exceptions. Refactoring these queries resolves the issue and returns proper response objects.
> 
> Suggestions:
> 
> Where appropriate, large tickets should be broken into subtasks to manage complexity and reviewability.
> 
> It may help to have a dedicated developer-only Slack channel for technical discussions. This could complement other community spaces if there’s a need to keep certain conversations more focused.
> 
> What are your thoughts on the above?
> 
> Thanks,
> Kapil