dimanche 19 avril 2015

Why does the performance lack in the Bitmap Index Scan?

I have the following sub query that is taking quite some time to execute.


I also put the EXPLAIN ANALYZE at http://ift.tt/1aJnpPe It seems it takes longest at the Bitmap Index Scan.


Can anyone provide some pointers on how to optimize this query.



foobar_production=> EXPLAIN ANALYZE SELECT
foobar_production-> date_trunc('day', c.time_at) AS time_at,
foobar_production-> COUNT(*) as count
foobar_production-> FROM (
foobar_production(> SELECT
foobar_production(> ds.user_id,
foobar_production(> max(case when id=maxid then country_id end) AS country_id,
foobar_production(> max(case when id=maxid then time_at end) AS time_at,
foobar_production(> max(case when id=maxid then properties->'assumed_gender' end) AS assumed_gender,
foobar_production(> json_agg(to_json(attachment_id)) AS attachment_ids
foobar_production(> FROM daily_statistics ds JOIN (
foobar_production(> SELECT u.id as user_id, (
foobar_production(> SELECT ds2.id FROM daily_statistics ds2 WHERE ds2.user_id=u.id AND ds2.metric = 'participation' AND ds2.status = 'active' AND ds2.campaign_id = 4742
foobar_production(> ORDER BY ds2.id DESC LIMIT 1
foobar_production(> ) AS maxid FROM users u
foobar_production(> WHERE u.properties -> 'provider' IN ('twitter')
foobar_production(> ) mu ON (ds.user_id=mu.user_id)
foobar_production(> WHERE ds.campaign_id = 4742 AND ds.metric = 'participation' AND ds.status = 'active'
foobar_production(> GROUP BY ds.user_id
foobar_production(> ) AS c
foobar_production-> GROUP BY date_trunc('day', c.time_at)
foobar_production-> ORDER BY time_at DESC
foobar_production-> LIMIT 7;


QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=2059.51..2059.54 rows=1 width=8) (actual time=19557.814..19558.392 rows=7 loops=1)
-> GroupAggregate (cost=2059.51..2059.54 rows=1 width=8) (actual time=19557.807..19558.377 rows=7 loops=1)
-> Sort (cost=2059.51..2059.52 rows=1 width=8) (actual time=19557.786..19558.070 rows=374 loops=1)
Sort Key: (date_trunc('day'::text, c.time_at))
Sort Method: quicksort Memory: 60kB
-> Subquery Scan on c (cost=2059.48..2059.50 rows=1 width=8) (actual time=19554.953..19557.062 rows=751 loops=1)
-> HashAggregate (cost=2059.48..2059.49 rows=1 width=103) (actual time=19554.919..19555.804 rows=751 loops=1)
-> Nested Loop (cost=0.98..1880.45 rows=2 width=103) (actual time=0.725..229.637 rows=2721 loops=1)
-> Index Scan using index_daily_statistics_on_campaign_id_and_type on daily_statistics ds (cost=0.56..1618.19 rows=31 width=99) (actual time=0.168..25.245 rows=9557 loops=1)
Index Cond: (campaign_id = 4742)
Filter: (((metric)::text = 'participation'::text) AND ((status)::text = 'active'::text))
-> Index Scan using index_users_on_id_and_type on users u (cost=0.42..8.45 rows=1 width=4) (actual time=0.014..0.015 rows=0 loops=9557)
Index Cond: (id = ds.user_id)
Filter: ((properties -> 'provider'::text) = 'twitter'::text)
Rows Removed by Filter: 1
SubPlan 1
-> Limit (cost=29.83..29.83 rows=1 width=4) (actual time=2.438..2.439 rows=1 loops=2721)
-> Sort (cost=29.83..29.83 rows=1 width=4) (actual time=2.433..2.433 rows=1 loops=2721)
Sort Key: ds2.id
Sort Method: top-N heapsort Memory: 25kB
-> Bitmap Heap Scan on daily_statistics ds2 (cost=25.80..29.82 rows=1 width=4) (actual time=1.890..2.213 rows=107 loops=2721)
Recheck Cond: ((user_id = u.id) AND (campaign_id = 4742))
Filter: (((metric)::text = 'participation'::text) AND ((status)::text = 'active'::text))
-> BitmapAnd (cost=25.80..25.80 rows=1 width=0) (actual time=1.879..1.879 rows=0 loops=2721)
-> Bitmap Index Scan on index_daily_statistics_on_user_id (cost=0.00..5.60 rows=156 width=0) (actual time=0.076..0.076 rows=297 loops=2721)
Index Cond: (user_id = u.id)
-> Bitmap Index Scan on index_daily_statistics_on_campaign_id_and_type (cost=0.00..19.95 rows=452 width=0) (actual time=1.770..1.770 rows=9557 loops=2721)
Index Cond: (campaign_id = 4742)
SubPlan 2
-> Limit (cost=29.83..29.83 rows=1 width=4) (actual time=2.320..2.321 rows=1 loops=2721)
-> Sort (cost=29.83..29.83 rows=1 width=4) (actual time=2.318..2.318 rows=1 loops=2721)
Sort Key: ds2_1.id
Sort Method: top-N heapsort Memory: 25kB
-> Bitmap Heap Scan on daily_statistics ds2_1 (cost=25.80..29.82 rows=1 width=4) (actual time=1.859..2.112 rows=107 loops=2721)
Recheck Cond: ((user_id = u.id) AND (campaign_id = 4742))
Filter: (((metric)::text = 'participation'::text) AND ((status)::text = 'active'::text))
-> BitmapAnd (cost=25.80..25.80 rows=1 width=0) (actual time=1.838..1.838 rows=0 loops=2721)
-> Bitmap Index Scan on index_daily_statistics_on_user_id (cost=0.00..5.60 rows=156 width=0) (actual time=0.092..0.092 rows=297 loops=2721)
Index Cond: (user_id = u.id)
-> Bitmap Index Scan on index_daily_statistics_on_campaign_id_and_type (cost=0.00..19.95 rows=452 width=0) (actual time=1.713..1.713 rows=9557 loops=2721)
Index Cond: (campaign_id = 4742)
SubPlan 3
-> Limit (cost=29.83..29.83 rows=1 width=4) (actual time=2.309..2.310 rows=1 loops=2721)
-> Sort (cost=29.83..29.83 rows=1 width=4) (actual time=2.300..2.300 rows=1 loops=2721)
Sort Key: ds2_2.id
Sort Method: top-N heapsort Memory: 25kB
-> Bitmap Heap Scan on daily_statistics ds2_2 (cost=25.80..29.82 rows=1 width=4) (actual time=1.802..2.069 rows=107 loops=2721)
Recheck Cond: ((user_id = u.id) AND (campaign_id = 4742))
Filter: (((metric)::text = 'participation'::text) AND ((status)::text = 'active'::text))
-> BitmapAnd (cost=25.80..25.80 rows=1 width=0) (actual time=1.785..1.785 rows=0 loops=2721)
-> Bitmap Index Scan on index_daily_statistics_on_user_id (cost=0.00..5.60 rows=156 width=0) (actual time=0.073..0.073 rows=297 loops=2721)
Index Cond: (user_id = u.id)
-> Bitmap Index Scan on index_daily_statistics_on_campaign_id_and_type (cost=0.00..19.95 rows=452 width=0) (actual time=1.680..1.680 rows=9557 loops=2721)
Index Cond: (campaign_id = 4742)
Total runtime: 19558.785 ms
(55 rows)


EDIT. Here is the table structure (columns and indices):



foobar_production=> \d daily_statistics;
Table "public.daily_statistics"
Column | Type | Modifiers
---------------+-----------------------------+---------------------------------------------------------------
id | integer | not null default nextval('daily_statistics_id_seq'::regclass)
type | character varying(255) |
metric | character varying(255) |
campaign_id | integer |
user_id | integer |
country_id | integer |
attachment_id | integer |
time_at | timestamp without time zone |
properties | hstore |
status | character varying(255) | default 'active'::character varying
Indexes:
"daily_statistics_pkey" PRIMARY KEY, btree (id)
"index_daily_statistics_on_attachment_id" btree (attachment_id)
"index_daily_statistics_on_campaign_id_and_type" btree (campaign_id, type)
"index_daily_statistics_on_country_id" btree (country_id)
"index_daily_statistics_on_id" btree (id)
"index_daily_statistics_on_metric" btree (metric)
"index_daily_statistics_on_properties" gin (properties)
"index_daily_statistics_on_status" btree (status)
"index_daily_statistics_on_time_at" btree (time_at)
"index_daily_statistics_on_user_id" btree (user_id)

Aucun commentaire:

Enregistrer un commentaire