REVIEW-11: SDIACT-200 Improve data transfer performance with a central logging and transfer analysis service - Design/Security Review

General design and security risk review of a new and permanent central logging and transfer performance analysis service

Concern 1: Detail was requested on specific fields to be stored in the database and on the REST API referenced in the document. (JP Navarro, Jim Basney)

• Resolution: List of fields from the log file has been added to the design document, and the REST API component of the design has been removed.

Concern 2: Questions were raised about the security implications of remote access to the database and other processing/data submission issues. (Jim Basney, Dave Carver)

• Resolution: Subsequent design modifications at the request of SPs have eliminated the need for remote SQL access, as log files are now copied directly to the central database location and processed centrally. Further clarification of SPs responsibility for scrubbing logs, if desired, was added to the design document. The desirability of a consistent policy for collecting and dissemination service usage information was expressed in the design review but is outside the scope of this activity.

Concern 3: Questions about specific performance analysis and other intended usages of the collected data. (JP Navarro, Dave Carver)

• Resolution: The design document was slightly altered to make it clearer that the primary usage of the database is to generate metrics and reporting information for NSF and internal usage, and to clarify the listed example(s) of potential performance analysis activities.

https://software.xsede.org/svn/sdi/activities/sdiact-200/trunk/Plans/SDIACT200-DSR-V5.pdf

https://software.xsede.org/svn/sdi/activities/sdiact-200/trunk/Plans/SDIACT-200-DSR.pdf

Criteria 1: Is the proposed data transfer information/logging that will be collected appropriate and complete to achieve the desired goals?

Criteria 2: Is the central data transfer information/logging repository design appropriate for the types of analysis and reporting that XSEDE needs?

Criteria 3: Are the security mechanisms appropriate for information in transit, stored in the central repository, and when accessed by information consumers?