|View Issue Details|
|ID||Project||Category||View Status||Date Submitted||Last Update|
|0001239||Pending Requests||Projects||public||2013-01-04 00:13||2016-12-26 10:25|
|Target Version||Fixed in Version|
|Summary||0001239: Creating or editing a project with a large number of existing projects is very slow or fails altogether|
|Description||Part of this problem is down to the PHP execution time-out being hit, failing the creation of a project. |
We have in excess of 10,000 projects in web2project. Each one is small - it's a potential project until it becomes a proper job. It's arguable that we should be running the "Helpdesk" plugin and only creating projects when we get a contract, but that's not my call. I suspect that it's the sheer number of projects/tasks being processed that's the problem, so it may not actually make a lot of difference.
Creating a project has got progressively slower and we had creation times of more than 45 seconds and I have had to increase the PHP script execution timeout to allow the process to complete. I eventually moved from a dual-core Pentium server to a 12-core Zeon LAMP server, (Ubuntu 12.03LTS, 32GB RAM) and got the creation time down to 20 seconds or so. That's shaved 2/3 of the execution time off, but the server still pedals hard when a "New Project" button is pressed.
Running TOP shows Apache running at 100% whilst a project is being created, but I'm pretty sure this is the PHP process. Running w2p with debug stats shows the database access times are adding milliseconds to the total.
Is there anything that can be done, other than caching, database optimisation and table analysis, which I run automatically every night. It looks like w2p is simply taking a hell of a lot of something and chewing it over before building the new project page. I can't see that it would need to select the entire contents of the database to create a new project, though.
|Additional Information||Running the same installation with an empty database is very, very fast.|
|Tags||No tags attached.|
When you create a project, it gets a list of all the projects in order to populate the "Parent Project" box. Even worse, it does a couple iterations i PHP to create the parent/child structures. If you have 10k projects, I'd wager that is 99% of the problem.
To resolve this one, we could:
- If you're using Active/Inactive projects consistently, we could limit the dropdown just to Actives.
- If you're not using parent/child projects, you could simply replace that database query with an empty array.
Would either of those work for you?
A bit more background: We're logging coordination requests for potential wind farms against our telecomms links, and we get dozens of these a day. Obviously, most of these farms won't get built (or we'd sink under the weight) but a small proportion of these requests turn into full-blown projects. We use the web2project project number as the master reference, so that's why they all start as projects. The sites and developers change names all the time, so that number is the only constant. The windfarms team moved from a spreadsheet to web2project, and tend to see it as a dialogue-based way of entering the spreadsheet data. They have little understanding of the capabilities of the tool they use.
I would say that the option of only loading active projects is the way to go for web2project, but for my particular application, I would prefer to be able to switch off that query!
I have two servers running web2project, sharing a single database, so I can switch off the query on the windfarms machine and leave the other one unchanged.
Are you considering a tick-box in the admin pages, or should I simply comment out something in the code? Either is fine by me. ;-)
Not sure on what the implementation will look like, but here's a work around.
Find this line in ./modules/projects/addedit.php:
$structprojs = $project->getAllowedProjects($AppUI->user_id, false);
and replace it with:
$structprojs = array();
That should do it for you.
|Thank you. It does. :-)|
There are two places this is probably causing problems..
First is the project import list. This should no longer be an issue because only projects marked as templates should be in the list.
Second is the project parent dropdown. This one still needs to be fixed.
|I was just thinking.. should the Parent Project dropdown default to the existing parent/tree plus all active projects? Or should it always display all the projects?|
|2013-01-04 00:13||simonnzg||New Issue|
|2013-01-04 09:09||caseydk||Note Added: 0002764|
|2013-01-04 09:09||caseydk||Status||new => feedback|
|2013-01-05 01:49||simonnzg||Note Added: 0002767|
|2013-01-05 11:12||caseydk||Note Added: 0002768|
|2013-01-05 11:12||caseydk||Project||v3.0 Release => Pending Requests|
|2013-01-05 14:47||simonnzg||Note Added: 0002769|
|2014-01-01 16:48||caseydk||Project||Pending Requests => v3.2 Release|
|2014-01-01 17:22||caseydk||Note Added: 0003166|
|2014-01-21 17:58||caseydk||Target Version||=> 3.2|
|2014-01-25 00:41||caseydk||Category||General => Projects|
|2014-03-27 09:44||caseydk||Project||v3.2 Release => v4.0 Release (Planning)|
|2014-06-10 22:45||caseydk||Note Added: 0003434|
|2014-06-10 22:49||caseydk||Target Version||3.2 => 4.0|
|2016-12-26 10:25||caseydk||Project||v4.0 Release (Planning) => Pending Requests|