You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@fineract.apache.org by "edk12564 (via GitHub)" <gi...@apache.org> on 2026/07/29 06:27:39 UTC
[PR] Implement Audit feature and Application Rate Limiting [fineract-consumer-facing]
edk12564 opened a new pull request, #78:
URL: https://github.com/apache/fineract-consumer-facing/pull/78
Issues: https://github.com/apache/fineract-consumer-facing/issues/65, https://github.com/apache/fineract-consumer-facing/issues/58
# Summary
Audit
With the new BFF and frontend layers, we have added new API surfaces that require auditing for better observability and compliance. For deployers in various countries, auditing configurations are set to a configurable amount in application.properties.
Architecturally, client-side auditing was left synchronous. This is in line with Fineract's existing auditing infrastructure. However, server-side (BFF-side) auditing was implemented using Spring EventBus to be handled asynchronously. This was done to improve performance on the frontend.
Additionally, for audit events that are required to be transactional, we used Spring Modulith's existing implementation that holds events in a transactional outbox so events are not lost. This ensures that the callsite and event live or die together. For nontransactional events, we simply add to an EventBus to be handled in memory.
Frontend audit requests are handled in batches with a time limit and batchsize limit.
Rate limiting is bundled with this change because audit allows a surface that attackers can exploit repeatedly, since an attack that creates many audit events can overload the database and event queue. Therefore, we have added rate limiting in the application layer that limits by user to a configurable amount in application.properties.
While this prevents authenticated users from overloading the database, unauthenticated endpoints like login and registration can still be overwhelmed. It is highly recommended that production environments use cloud deployment practices to take advantage of firewalls, DDoS protection, and network rate limiting to protect these endpoints.
Finally, existing features in both the frontend and backend were modified to call these auditing endpoints/publishers.
## More Details
- implemented audit feature
- added event driven architecture with EventBus for server side audits
- utilized Spring Modulith to handle transactional events
- added liquibase migration for nontransactional audit events
- added liquibase migration for Spring Modulith's transactional events
- added PIIScreen for information in the audit table to protect user data
- implemented @TransactionalAuditListener and @NonTransactionalAuditListener aspects to handle different types of audits
- separated serverside and clientside audit entries in table
- added configurable audit table TTL for serverside vs clientside entries
- called clientside audit events in batches on the frontend
- implemented rate limiting per user
- added Redis rate limit cache to handle rate limiting between server instances
- added cucumber and unit tests to confirm rate limiting functionality
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fineract.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement Audit feature and Application Rate Limiting [fineract-consumer-facing]
Posted by "Aman-Mittal (via GitHub)" <gi...@apache.org>.
Aman-Mittal commented on code in PR #78:
URL: https://github.com/apache/fineract-consumer-facing/pull/78#discussion_r3683241502
##########
consumer/src/main/java/org/apache/fineract/consumer/loans/command/service/LoansCommandServiceImpl.java:
##########
@@ -76,7 +74,6 @@ public LoanApplicationCommandData submitApplication(Jwt jwt, SubmitLoanApplicati
}
@Override
- @Command
public LoanApplicationCommandData modifyApplication(Jwt jwt, ModifyLoanApplicationCommand command) {
Review Comment:
Why removed command?
##########
consumer/src/main/java/org/apache/fineract/consumer/audit/command/repository/AuditEventCommandRepository.java:
##########
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.fineract.consumer.audit.command.repository;
+
+import java.time.Instant;
+import org.apache.fineract.consumer.audit.command.domain.AuditEvent;
+import org.springframework.data.jpa.repository.JpaRepository;
+import org.springframework.data.jpa.repository.Modifying;
+import org.springframework.data.jpa.repository.Query;
+import org.springframework.data.repository.query.Param;
+import org.springframework.transaction.annotation.Transactional;
+
+public interface AuditEventCommandRepository extends JpaRepository<AuditEvent, Long> {
+
+ @Transactional
+ @Modifying
+ @Query(nativeQuery = true, value = "DELETE FROM audit_events WHERE id IN "
+ + "(SELECT id FROM audit_events WHERE source = :source AND received_at < :cutoff LIMIT :batchSize)")
Review Comment:
use @Query(
nativeQuery = true,
value = """
DELETE FROM audit_events
WHERE id IN (
SELECT id
FROM audit_events
WHERE source = :source
AND received_at < :cutoff
LIMIT :batchSize
)
"""
)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fineract.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement Audit feature and Application Rate Limiting [fineract-consumer-facing]
Posted by "edk12564 (via GitHub)" <gi...@apache.org>.
edk12564 commented on code in PR #78:
URL: https://github.com/apache/fineract-consumer-facing/pull/78#discussion_r3684744168
##########
consumer/src/main/java/org/apache/fineract/consumer/loans/command/service/LoansCommandServiceImpl.java:
##########
@@ -76,7 +74,6 @@ public LoanApplicationCommandData submitApplication(Jwt jwt, SubmitLoanApplicati
}
@Override
- @Command
public LoanApplicationCommandData modifyApplication(Jwt jwt, ModifyLoanApplicationCommand command) {
Review Comment:
Hi Aman, I removed this and @Query because the original purpose was to use for auditing. But I think the Transactional/NonTransactional on listeners boundary is better for this. So it shouldn't be necessary anymore.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fineract.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement Audit feature and Application Rate Limiting [fineract-consumer-facing]
Posted by "edk12564 (via GitHub)" <gi...@apache.org>.
edk12564 commented on code in PR #78:
URL: https://github.com/apache/fineract-consumer-facing/pull/78#discussion_r3684765023
##########
consumer/src/main/java/org/apache/fineract/consumer/audit/command/repository/AuditEventCommandRepository.java:
##########
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.fineract.consumer.audit.command.repository;
+
+import java.time.Instant;
+import org.apache.fineract.consumer.audit.command.domain.AuditEvent;
+import org.springframework.data.jpa.repository.JpaRepository;
+import org.springframework.data.jpa.repository.Modifying;
+import org.springframework.data.jpa.repository.Query;
+import org.springframework.data.repository.query.Param;
+import org.springframework.transaction.annotation.Transactional;
+
+public interface AuditEventCommandRepository extends JpaRepository<AuditEvent, Long> {
+
+ @Transactional
+ @Modifying
+ @Query(nativeQuery = true, value = "DELETE FROM audit_events WHERE id IN "
+ + "(SELECT id FROM audit_events WHERE source = :source AND received_at < :cutoff LIMIT :batchSize)")
Review Comment:
Should be fixed in my most recent commit!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fineract.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement Audit feature and Application Rate Limiting [fineract-consumer-facing]
Posted by "edk12564 (via GitHub)" <gi...@apache.org>.
edk12564 commented on code in PR #78:
URL: https://github.com/apache/fineract-consumer-facing/pull/78#discussion_r3684744168
##########
consumer/src/main/java/org/apache/fineract/consumer/loans/command/service/LoansCommandServiceImpl.java:
##########
@@ -76,7 +74,6 @@ public LoanApplicationCommandData submitApplication(Jwt jwt, SubmitLoanApplicati
}
@Override
- @Command
public LoanApplicationCommandData modifyApplication(Jwt jwt, ModifyLoanApplicationCommand command) {
Review Comment:
Hi Aman, I removed it because the original purpose was to use for auditing. But I think the Transactional/NonTransactional on listeners boundary is better for this. So it shouldn't be necessary anymore.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fineract.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement Audit feature and Application Rate Limiting [fineract-consumer-facing]
Posted by "edk12564 (via GitHub)" <gi...@apache.org>.
edk12564 commented on code in PR #78:
URL: https://github.com/apache/fineract-consumer-facing/pull/78#discussion_r3684744168
##########
consumer/src/main/java/org/apache/fineract/consumer/loans/command/service/LoansCommandServiceImpl.java:
##########
@@ -76,7 +74,6 @@ public LoanApplicationCommandData submitApplication(Jwt jwt, SubmitLoanApplicati
}
@Override
- @Command
public LoanApplicationCommandData modifyApplication(Jwt jwt, ModifyLoanApplicationCommand command) {
Review Comment:
Hi Aman, I removed this and @Query because the original purpose was to use for auditing. But I think the Transactional/NonTransactional on listeners boundary is better for this. So it shouldn't be necessary anymore. There might be some use for it down the line, but right now the annotation is a no-op.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fineract.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement Audit feature and Application Rate Limiting [fineract-consumer-facing]
Posted by "Aman-Mittal (via GitHub)" <gi...@apache.org>.
Aman-Mittal merged PR #78:
URL: https://github.com/apache/fineract-consumer-facing/pull/78
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fineract.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org