Anonymous Login
2019-09-22 20:47 PDT

View Issue Details Jump to Notes ]
IDProjectCategoryView StatusLast Update
0001239Pending RequestsProjectspublic2016-12-26 10:25
Reportersimonnzg 
Assigned To 
PrioritynormalSeverityminorReproducibilityalways
StatusfeedbackResolutionopen 
Product Version 
Target VersionFixed in Version 
Summary0001239: Creating or editing a project with a large number of existing projects is very slow or fails altogether
DescriptionPart of this problem is down to the PHP execution time-out being hit, failing the creation of a project.

We have in excess of 10,000 projects in web2project. Each one is small - it's a potential project until it becomes a proper job. It's arguable that we should be running the "Helpdesk" plugin and only creating projects when we get a contract, but that's not my call. I suspect that it's the sheer number of projects/tasks being processed that's the problem, so it may not actually make a lot of difference.

Creating a project has got progressively slower and we had creation times of more than 45 seconds and I have had to increase the PHP script execution timeout to allow the process to complete. I eventually moved from a dual-core Pentium server to a 12-core Zeon LAMP server, (Ubuntu 12.03LTS, 32GB RAM) and got the creation time down to 20 seconds or so. That's shaved 2/3 of the execution time off, but the server still pedals hard when a "New Project" button is pressed.

Running TOP shows Apache running at 100% whilst a project is being created, but I'm pretty sure this is the PHP process. Running w2p with debug stats shows the database access times are adding milliseconds to the total.

Is there anything that can be done, other than caching, database optimisation and table analysis, which I run automatically every night. It looks like w2p is simply taking a hell of a lot of something and chewing it over before building the new project page. I can't see that it would need to select the entire contents of the database to create a new project, though.
Additional InformationRunning the same installation with an empty database is very, very fast.
TagsNo tags attached.
Attached Files

-Relationships
+Relationships

-Notes

~0002764

caseydk (administrator)

When you create a project, it gets a list of all the projects in order to populate the "Parent Project" box. Even worse, it does a couple iterations i PHP to create the parent/child structures. If you have 10k projects, I'd wager that is 99% of the problem.

To resolve this one, we could:
- If you're using Active/Inactive projects consistently, we could limit the dropdown just to Actives.
- If you're not using parent/child projects, you could simply replace that database query with an empty array.

Would either of those work for you?

~0002767

simonnzg (reporter)

A bit more background: We're logging coordination requests for potential wind farms against our telecomms links, and we get dozens of these a day. Obviously, most of these farms won't get built (or we'd sink under the weight) but a small proportion of these requests turn into full-blown projects. We use the web2project project number as the master reference, so that's why they all start as projects. The sites and developers change names all the time, so that number is the only constant. The windfarms team moved from a spreadsheet to web2project, and tend to see it as a dialogue-based way of entering the spreadsheet data. They have little understanding of the capabilities of the tool they use.

I would say that the option of only loading active projects is the way to go for web2project, but for my particular application, I would prefer to be able to switch off that query!

I have two servers running web2project, sharing a single database, so I can switch off the query on the windfarms machine and leave the other one unchanged.

Are you considering a tick-box in the admin pages, or should I simply comment out something in the code? Either is fine by me. ;-)

~0002768

caseydk (administrator)

Not sure on what the implementation will look like, but here's a work around.

Find this line in ./modules/projects/addedit.php:
$structprojs = $project->getAllowedProjects($AppUI->user_id, false);

and replace it with:
$structprojs = array();

That should do it for you.

~0002769

simonnzg (reporter)

Thank you. It does. :-)

~0003166

caseydk (administrator)

There are two places this is probably causing problems..

First is the project import list. This should no longer be an issue because only projects marked as templates should be in the list.

Second is the project parent dropdown. This one still needs to be fixed.

~0003434

caseydk (administrator)

I was just thinking.. should the Parent Project dropdown default to the existing parent/tree plus all active projects? Or should it always display all the projects?
+Notes

-Issue History
Date Modified Username Field Change
2013-01-04 00:13 simonnzg New Issue
2013-01-04 09:09 caseydk Note Added: 0002764
2013-01-04 09:09 caseydk Status new => feedback
2013-01-05 01:49 simonnzg Note Added: 0002767
2013-01-05 11:12 caseydk Note Added: 0002768
2013-01-05 11:12 caseydk Project v3.0 Release => Pending Requests
2013-01-05 14:47 simonnzg Note Added: 0002769
2014-01-01 16:48 caseydk Project Pending Requests => v3.2 Release
2014-01-01 17:22 caseydk Note Added: 0003166
2014-01-21 17:58 caseydk Target Version => 3.2
2014-01-25 00:41 caseydk Category General => Projects
2014-03-27 09:44 caseydk Project v3.2 Release => v4.0 Release (Planning)
2014-06-10 22:45 caseydk Note Added: 0003434
2014-06-10 22:49 caseydk Target Version 3.2 => 4.0
2016-12-26 10:25 caseydk Project v4.0 Release (Planning) => Pending Requests
+Issue History